COMPUTATIONAL 
METHODS OF 
LINEAR ALGEBRA 


By 
V. N. FADDEEVA 


Authorized translation from the Russian by 
Curtis D. Benster 


Dover Publications, Inc. 


New York 


AUTHOR’S PREFACE 


The numerical solution of the problems of mathematical 
physics is most frequently connected with the numcrical solution of 
basic problems of linear algebra—that of solving a system of linear 
equations, and that of the computation of the proper numbers of a 
matrix. The present book is an endeavor at systematizing the most 
important numerical methods of linear algebra—classical ones and 
those developed quite recently as well. 

The author docs not pretend to an exhaustive completeness, hav- 
ing included an exposition of those methods only that have already 
been tested in practice. In the exposition the author has not strived 
for an irreproachable rigor, and has not analysed all conceivable 
cases and sub-cases arising in the application of this or that method, 
having limited herself to the most typical and practical important 
cases. 

The book consists of three chapters. In the first chapter is given 
the material from linear algebra that is indispensable to what fol- 
lows. The second chapter is devoted to the numcrical solution of 
systems of linear equations and parallel questions. Lastly, the third 
chapter contains a description of numerical methods of computing 
the proper numbers and proper vectors of a matrix. 

For the interest manifested in the manuscript, and for a number 
of valuable suggestions, I express my sincere thanks to A. L. Brudno 


and G. P. Akilov. 
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TRANSLATOR’S NOTE 


But for the initiative of Dr. George E. Forsythe, this trans- 
lation would not have been written; it is thanks to his awareness 
and appreciation of this work in the original, as well as to his support 
of the translating, that this English-language version now appears. 
And to Mrs, Faddeeva, the author, go my respects for a real com- 
putor’s guide-book, with a nice sense of both theory and practice, 
and a presentation no less nice—a book such as I wish I had had. 

Grateful acknowledgments are duc to my father, L. Halsey 
Benster, for contributions beyond a mechanical engincer’s line of 
duty (including proofreading in particular), and to my wife Ada, 
for more than can be expressed. June Wolfenbarger’s fastidious 
typing helped get the manuscript off the ground. 

I have added a few notes [those in brackets], replaced many of 
the Russian references with more accessible ones, and re-computed 
all of the principal tables, which I hope will thus be especially 
reliable and useful guides. 

Curtis D. BENSTER 
Ophir, Colorado 
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COMPUTATIONAL 
METHODS OF 
LINEAR ALGEBRA 


CHAPTER I 


BASIC MATERIAL FROM LINEAR 
ALGEBRA 


This chapter will be of an auxiliary character. Without 
detailed proofs, it will impart material from lincar algebra that 
will be indispensable to an understanding of the following chapters. 


§ 1. MATRICES 


1. An aggregate of numbers—which are, generally speaking, 
complex—arranged in the form of a rectangular table, is called a 
rectangular matrix. This array will have m rows and n columns, and 
may be set forth in the form: 


5 0 ec Qin 
Qo) Qov eee Go 

(1) ef = sy ’ 
| Aye oes Grn 


the first subscript, then, designating the row, the second designating 
the column, in which the element in point is located. 
This may be abbreviated to the form: 


A=[a,;) (§=1,2,...,mj;j=1,2,...,2). 


Two matrices are equal if their corresponding elements are equal. 
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Matrices composed of a single row are called simply rows (or, as 
they will be approached later, row vectors). Matrices composed of 
a single column are called columns (or column vectors). 

If the number of rows of a matrix equals the number, 2, of 
columns, it is called square, and of the nth order. 

Among square matrices, an important role is played by diagonal 
matrices, i.e., matrices of which only the elements along the prin- 
cipal (leading) diagonal are different from zero: 


a 0 eee 0 
0 s ae O 

(2) ne = Ta, Qo . +5 pj. 
0 0 ... @ 


If all the numbers a; of such a matrix are equal to each other, the 
matrix is said to be scalar: 


0 


0 
(3) a ... O| = lay 
0 


Oo © R&R 


0 4 


and, ifa=1, the matrix is said to be the unté matrix: 


10... 0 
O11... 0 

(4) =]. 
00... 1 


Lastly, a matrix all of whose elements are cqual to zero is called a 
null matrix, or zero matrix. We shall designate it by the symbol 0. 

The determinant whose elements are the elements of a square 
matrix (without disarrangement) is said to be the determinant of that 
matrix, and we write the determinant of the matrix 4 as | A], or often 
as d(A). 

2. Multiplication of a matrix by a number. The addition of matrices. 
A matrix whose elements are obtained by multiplying all the ele- 
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ments of the matrix A by a number a is called the product of the 


number «@ and the matrix 4: 


Aay, 2a,» aa;,, 

QAv, Maw» Maw 
(5) aA = is 

aa. 242 Qa nn 


A matrix C whose elements are the sums of the corresponding ele- 
ments of A and B, matrices having like numbers of rows and columns, 
is called the sum of A and B: 


Ayt5y)  ay2+by. a,,+ 51, 
6) Aa Ba | Uther Geet bes ++ tant be 
Ami + bet ano + Bins: Bian + Bann 


The operations introduced above 
as will be readily seen: 


have the following properties, 


lh A+(B4+C) = (A+ B)+4+C. 
2, A+B = B+A, 

3. +0 = A. 

4. (a+fB)A = cA+fa. 

59, @(A+B) = cA+aBb. 


Here A, B, and C are matrices; @ and f are numbers—generally 
speaking, complex. 

3. The multiplication of matrices. Multiplication of the matrices A 
and B is defined only on the assumption that the number of columns 
of matrix A equals the number of rows of matrix B. On this 
assumption, the elements of the product, C=AB, are defined in the 
following manner: the element in the ith row and the jth column 
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of the matrix C is equal to the sum of the products of the elements 
of the :th row of the matrix A by the corresponding elements of 
the jth column of matrix B.!— Thus: 


411 419 a1, bi, dyn... b1, 
(7) AB= Qo, fog «+e ay bay bay... day 
Gn) ane Ginn bay Bao oe bap 
11 32 ~- 1p 
_| S21 ‘22 C2p 2G 
Cnt Oma cc+ Cmp 
where 


n 
Cig = 4; ;+4j9b0;+ +++ +4indg, = Pa a,b, , 


(8) 
(i=1,2,...,m;j=1,2,...,p). 


It is to be noted that the product of two rectangular matrices is 
again a rectangular matrix, the number of rows of which is equal to 
the number of rows of the first matrix, and the number of columns 
of which is equal to the number of columns of the second matrix: 


mxn axp mx 


mm C. So, for instance, the product of a square matrix and 
a matrix composed of one column is a matrix of one column. 

The commutative law for multiplication does not, generally 
speaking, hold. We shall make a few observations on this subject, 
however. The matrices AB and BA make sense simultaneously only 
if the number of rows of the first matrix is equal to the number of 
columns of the second, and the number of columns of the first is 
equal to the number of rows of the second. Given the fulfillment 
of these conditions, the matrices AB and BA will both be square, but 
of different orders, unless A and B be square. ‘Thus even to put the 

1 [As to the motive for this apparently arbitrary definition, sce, e.g., AITKEN, 


A. C.,» [2], § 3. Abundant numerical illustrations will be found in Frazer, R. A., 
Duncan, W. J., anD Coirar, A. R.,[1]}, § 1.4.] 
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question of the equality of the matrices AB and BA makes sense only 
for square matrices. But even in this case, generally speaking, 
ABABA. 

In particular cases multiplication may be commutative, and in 
such cases the matrices arc said to commute. Thus, for example, 
scalar matrices commute with any square matrix of the same order, 


for 
6 4 0 eee 0 Qi1 Qy\o oe Qin 
O@ 224.0 Go, fox ..- Gay 
(9) 
0 O a rr Pe 
Q)1 Ayo . Qin a 0 oem 0 
Qa, Goo .-+- a, a 0 
an) a,2 oes Quan 0 0 e e a 


Qa» Qaoo ee Aas, 
Qa} aa ,2 . Qa nn 


Hence follows the special role of the unit matrix in the multiplica- 
tion of matriccs, to wit: amongst all square matrices of the same 
order, the unit matrix plays the samc role as the number one does 
among numbers. Indecd, 


It can be shown that the multiplication of matrices is assoczative, 
viz.: if AB and (AB)C makc sense, so also do BC and A(BC}, and 


1. A(BC) = (ABC. 


6 Baste Material from Linear Algebra 
The matrix product has also these properties: 
2. a(AB) = (acA)B = A(aB); . 
3. (A+B)C = AC+BC; 
4. C(A+B) = CA+CB, 


where 4, B, C are matrices, a a number. 
Let us interchange rows and columns in the matrix 


Q@11 2 Q15 
@o1 4ee ao, 

A= = [a;;]; 
ani Ane ann 


yy Any ees Ant 
(10) AO ae, AT as [a; J’ = Q@yo 4ngq +e es Ang 2 [a;,]. 
Ain 42 ann 


The following rule (the reversal rule) for a transposed product 
should be noted: 


(AB)’ = B’A’ 

In proof of this, note that the clement of the ith row and jth 
column of the matrix (AB)’ is cqual to the element of the jth row 
and ith column of the matrix AB, for this is merely the interchange 
of row with column, i.e., transposition; and that is equal to 


(11) 05164; + joboj,+ + >> +4jq,;- 


The last expression is obviously equal to the sum of the products of 
the elements of the :th row of the matrix B’ and the corresponding 
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elements of the jth column of the matrix 41’, i.c., is equal to the 
general element of the matrix B's’. 

In conclusion we shall remark that the determinant of a product 
of square matrices is equal to the product of the determinants of the 
multiplied matrices: |AB|=|Aj |B], which result is taken from deter- 
minant theory. 

4. The partitioning of matrices. The handling of matrices of high 
orders requires, as a rule, a large number of operations. It is there- 
fore often convenient to reduce a computation involving matrices 
of high orders to computations upon matrices of lower orders. Such 
a reduction can be effected by partittoning the given matrices: each 
matrix may be conceived as composed of several matrices of lower 
orders, and this subdivision may be carried through in many ways, 
for example: 


4i1 232 &33 iy aiy ; Gye 413° «ang 

Qo} aay aoy Av, Peano ee ay oe eee sowie eeavntnevenan e 

Q3, @gyo 433 434 Gx, + Agy Gag Aggy 
a3) > Q@gy 833 434 
aiy aya 419 G14 


Go, Guys Ang Ang 


43} @yz > gg yg 

The matrices into which the given matrix is partitioned are called 
its submatrices, or cells. In such a partition the horizontal and ver- 
tical subdividing lines are of course supposed to be carried across the 
whole matrix. 

We shall not concern ourselves with the general case of the parti- 
tioning of a matrix,’ but shall here consider only a partition of 
square matrices in which the diagonal submatrices are square. 

The basic operations on partitioned matrices whose diagonal 
matrices are of identical orders is connected in a quite natural way 


1 [Sce, e.g., Arrken, A. C., [2], § 11.) 
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with operations upon the submatrices themselves. To wit, if we 
have 


iy Ayy Ay 
1, fou 1, 
A = i 2) a oe? f uy | 
Ay Aye Ay 
and 
By, Bip By 
By apep Bs 
B= a1 Bas 
Bi, Bye Buy 


where A;; and B;; are square matrices of the same order, then 
Ayt+Biy AywtBy -.. AytBy 
Agi + Ba, Aget Boy... Ant Bry 


(12) A+Be= 
Ant+By Apt Bye --- AntBu 

and 

Cry Cz Cy 

Co Coo oes Ci 
(13) AB | 7? re a 

Cyr Cy Cu 
where 


C;; = A,B, + Aj2.Bo,+ =< +A;,Bi; Gj=l, ese y k). 


We shall not stop for a proof of the last formula, but will note here 
only that the matrices A;,; and B,; can indeed be multiplied, since 
the number of columns of matrix A;,; equals the number of rows of 
the matrix By; 

Formulas (12) and (13) show that operations with matrices parti- 
tioned in the manner indicated are to be performed just as if in 
place of cach submatrix there was a number. 


9 


An important special case of a partitioned matrix is the bordered 
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matrix. Having a square matrix -1,_, of order n—1: 
@11 A120 cs Pe | 
ao Avy Ge, n-1 
A,-1 = ? 
Gn, 1 4,-1, eo oe ee And yl 


we form a square matrix of the nth order, A,, by appending to the 


matrix sl,.) a TOW: U,_1=(@g1,--+5 4%, n-1), @ column: 4u,_1 


= (4j,,) --+54,-1. ,), and a number a,,: 
a), 
Anat Gon 4 ‘ 
(14) A, = ; = |: “eA mt 
a1. ; “al mn 
OP a, n—1 Fn 


We shall say that the matrix 1, has been obtained by bordering the 
matrix sJ,_,.. “The matrix A, is naturally partitionable. 

Operations upon a bordered matrix are performed in accordance 
with the general rules for operations upon partitioned matrices. 


_ J. wef] 


be two bordered matrices of order n, the meaning of M, v, u, a, and 
P, x, y, 6, being those of the definition, the following statements are 
valid: 


P 


v 


y 
b 


lf 


i u 


v a 


( ey eT 

aA = ; 
av aa 
\f+P 

(15) ’ A+B = |! + aie 

U+X at+b 

‘B= boty My + ub 

LP bax vy + ab | 
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Here MP and ux are matrices of the (n—1)th order; My, ub are 
columns composed of 2—1 elements; vP and ax are analogous rows; 
and, lastly, vy +a is a number. 

5. Quasi-diagonal matrices. Let us consider still another particular 
case of partitioned matrices, namely the matrices called quasi- 
diagonal. These are square matrices along whose leading diagonals 
are arrayed square submatrices, the remainder of the elements being 
zero. An example would be the seventh-order quasi-diagonal 
matrix: 

44, ao: O OO QO : QO O 
ay, a2 : 0 0 0 : O O 


O O : by bye by : 0 O 
0 0 : bg, bye bg : O O 
0 0 0 0 0 C11 C12 
0 0 : 0 0 O ? Cy, eg 


The cells of this matrix are obviously 


a a 
Az 11 ati 
&o1 4229 


53) bso b33 


and the six null submatrices. 

If two quasi-diagonal matrices are of like structure, the product of 
such matrices will also be a quasi-diagonal matrix of the same struc- 
ture, the diagonal cells of which equal the products of the corrc- 
sponding cells of the factor matrices. 

The determinant ofa quasi-diagonal matrix is equal to the product 
of the determinants of the diagonal cells, on the strength of a notable 
theorem by Laplace.} 


1 [See, ¢.g., Arrxen, A. C., {2}, § 33.) 
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6. The inverse and the adjoint matrices. A square matrix A=(a;,] 1s 
said to be non-singular if its determinant is not equal to zero; in the 
contrary case it is of course singular. 

The important concept of an inverse matrix is now introduced. 
A matrix B is called the inverse (or reciprocal) of the matrix A if 


(16) AB = I. 


We shall show that the necessary and sufficient condition for the 
existence of the inverse matrix is the non-singularity of the matrix A. 

The necessity follows at once from the theorem concerning the 
determinant of a matrix product, for if AB=/J, |A| |B|=1 and con- 
sequently |A}| <0. 

Assume now that |A|]#0. In order to construct the inverse 
matrix we must give preliminary consideration to the adjoint (or 
adjugate) matrix, i.c., the matrix 


Aj; Ay, 2 ae An 


(17) on Ajo Ags : Ae 
Aj, Ay, Ann 


Here 4; ,; is the algebraic complement (cofactor) of the element a;; in 
the determinant of the matrix <A, i.ec., is the signed minor deter- 
minant of the element a;;._ Note that it is here placed in transposed 
position. 

We shall show that the adjoint matrix has the following property: 


(18) AC = |All 


In demonstration, reckoning the general element of the matrix AC 
by the rule for matrix multiplication, we find it to equal 


Qi Aj +ajeAjo+ -- + +4;,A jns 


i.e., zero for jy, and |A| for :=j, on the strength of a familiar 
theorem on the expansion of determinants.! 


1 (See, e.g., Arrxen, A. C., [2], § 21.) 
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The equality 
(18’) CA = |All 


is established in like manner. 
The adjoint matrix has meaning for any square matrix A. From 
the equality AC=|A|/ it follows that the matrix 


(19) B=7C 


is, for non-singular A, the sought inverse, for 


| 
AB=A—C=—ACH1. 
|4[ [al 


The constructed matrix has also the property 
(20) BA = 1, 


which follows from equation (18’). 

We prove, lastly, the uniqueness of the inverse matrix. Assume 
that a matrix X exists such that AX=/. Multiplying this equation 
by B on the left, we have X=B. If it be assumed that YA=/,a 
multiplication on the right by B yields Y= 8B. 

The matrix inverse to A is denoted by A-!. It is obvious that 
|A~] =|A]-?. 

We note that the inverse of the product of two matrices also 
displays the reversal rule: 


(21) (A,Aq)-! = Az dz, 
since 
A,AoAg! Aj! = A Ay! = 1. 


The determination of the inverse matrix is one of the fundamental 
problems of linear algebra. Equation (19) offers the possibility of 
computing the inverse matrix; however, the computation of the 
adjoint matrix is so labor-consuming that the cited equation is of 
importance only in theoretical relationships. Chapter II will be 
specially devoted to this problem of determining the inverse matrix. 
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7. Polynomials in a matrix. We now define the positive integral 
power of a square matrix, putting 


n times 


22 ie. 
ee A-A» ...°:A = Ar, 


In view of the associative law, how the parentheses in this product , 
are placed makes no difference, and we therefore omit them. It is 
evident from the definition that 


AnAn = Antm 
(23) { 


(An)}m = Anam, 


Hence it follows that powers of the same matrix are commutative. 
We further put, by definition, 


A® = ], 


An expression of the form 


agA*+a,Ar-!4 .-- +a,/, 


where ao, a), ...,@, are complex numbers, is called a polynomial in 
a matrix, or matrix polynomial. This matrix polynomial may be 
regarded as the result of replacing the variable 2 in an algebraic 
polynomial 


(24) p(A) = AoA" +o,As-14 Sige +a, 


by the matrix A. 

It is important to note that the rules for operation upon matrix 
polynomials do not differ from the rules for operation upon algebraic 
polynomials, viz.: 


given (A) = p(A) £x(A) 
(A) = wld) x(a), 

then g(a) = p(A)+7(A) 
w(A) = p(d) 7(A). 


This follows from the commutativity of the powers of a matrix. 


(25) 
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8. The characteristec polynomial. The Cayley-Hamilton theorem. The 
minimum polynomial. ‘The equation 


a,—A 419 see G1, 
(26) ay Goo—A pies a2, — 0 
a.) a,9 .2. A,-A 


is called the characteristic, or secular, equation of the matrix A=(a;,). 
The left member of this equation, which may be written in the 
abbreviated form |A —1J|, bears the name characteristic polynomial (or 
characteristic function) of the matrix. Characteristic equations are 
frequently encountered in applied mathematics. 

The direct computation of the characteristic function presents 
considerable technical difficulties. If 


(27) g(A) = |A—Al] = (—1)9[Ar— pide —pade2— «+» py), 
then 


fi = 2, +4Q2+ + °° +4,,, 
(28) 


pf, = (= 1)s-1|A|, 


and the remaining coefficients p, are the sums, taken with the sign 
(—1)4-!, of all the principal minors of the determinant of matrix A of 
order k, i.e., of the minors involving the principal diagonal.! The 
number of such minors equals the number of combinations of 2 
things taken & at a time. 

The roots of the characteristic equation are called the proper 
numbers (or characteristic roots, latent roots, proper values, eigenvalues) of 
the matrix A. From the well-known theorem of Vieta giving 
the connection between the roots of an equation and its coefficients, 
we have 


(29) { Aitagt ae +A, = PP) = 44; + Go0ot+ --- +4,, 
AyAg... A, = (—1)*"'p, = [Al. 
1 [See, e.g., Arrxen, A. C., [2], § 37.] 
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The quantity py =a,;+a@o0.+ ---+ +a,, is called the trace (or spur) 
of the matrix A, and is denoted by tr 41. 

Practically convenient methods for determining the coeflicients 
and roots of the characteristic equation will be elaborated in 
Chapter IIT, which will be specially devoted to that group of ques- 
tions. For the moment we Icave them aside. 

For any square matrix the following remarkable relation, known 
as the Cayley-Hamilton theorem, obtains: if g(A) is the characteristic 
polynomial of the matrix A, then g(A4)=0, that is, in a sense, the 
matrix is a rool af ils own characteristic equation. 

For proof, let us consider the matrix 8, the adjoint of the matrix 
A—Al, Since each cofactor in the determinant |A— //| is a poly- 
nomial in 4 of degree not exceeding n— 1, the adjoint matrix may be 
represented as an algebraic polynomial with matrix coefficients, 
i.c., in the form 


B= B, +B, dt +++ + Bode, 


where B,_;,..., Bg are certain matrices not dependent on A. On 
the strength of the fundamental property of the adjoint matrix, we 
have 


(B,_1+8,-oAt eee + Bods-!)(A—Al) = JA — All 
= (—I)*(An—p,An-1— --- —p, JI. 


This equation is equivalent to the system of cquations 
B,-1A a (- I)atlp 
B,-2A—- B,-1 - (= l)**1p,-al 
BoA-By = (=!) pl 
= Bo = ( = 1)ef 


Multiplying these equations on the right by /, A, A2,..., A»-!, Aa, 
respectively, and adding, we obtain a null matrix on the left side, 
and on the right, 


(30) (Vl -p,.f —pa-1A —py-2l? — -- - +A] = (A). 
Thus 9(A) =0, which is what was required to be proved. 
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The Cayley-Hamilton relation shows that for a given square 
matrix a polynomial exists for which it is a root. Evidently such a 
polynomial is not unique, for if y(A) has such a property, so has any 
polynomial] divisible by y(4). The polynomial of lowest degree 
having the property that the matrix A is a zero of it, is called the 
minimum polynomial of the matrix A. 

We shall prove that the characteristic polynomial is divisible by 
the minimum polynomial. 

Let g(4) and r(A4) be the quotient and remainder obtaincd upon 
dividing the characteristic polynomial @(4) by the minimum poly- 
nomial w(A): 


g(A) = (A) g(a) +7(A), 


the degree of r(A) being of course less than the degree of p(A). 
Substituting A for 4 in this equation, we have 


r(A) = 9(A) (A) g(A) = 0. 


Thus the matrix A proves to be a “zero” of the polynomial r(A); it 
thence follows that r(A) =0, since otherwise (A) would not be the 
minimum polynomial. Consequently (4) divides (A). 

9, Similar matrices. The matrix B is said to be similar to the 
matrix A if a non-singular matrix C exists such that B=C-lAC. 
Matrix B is said to be obtained from matrix A by a similarity trans- 
formation. 

The similarity transformation has the following properties: 


lh CoA, CHCHACH +++ +O714.0 
= C7'(A, + Agt Hoece +A,)C. 


(31) 2. C-'A,C-C-14eC +... C-1A,C = C-1(A, Ag... A,)C. 


In particular, (C-!AC)= = C-!AnC. 
Hence: 


3. f(C-1AC) = C-1 f(A) C for any polynomial f(A). 
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From the last property it follows directly that simular matrices have the 
same minimum function. 
We shall show that similar matrices have also the same charactertstic 
function. 
We have 


|B—Al| 


\C-1AC— All| = |C-1AC— AC-HC| 


IC)“ —Ad| |C] = Jaa]. 


10. dslementary transformations. tis frequently necessary to effect 
. the following operations upon matrices: 

a) Multiplication of the elements of some row by a number; 

b’) Adding to the elements of some row numbers proportional to 
the clements of some preceding row. 

b’) Adding to the clements of some row numbers proportional to 
the clements of some following row. 

Sometimes such transformations must be made upon the columns, 
Transformations of the type indicated are called elementary trans- 
JSormations of the matrix. 

Any elementary transformation of the rows is equivalent to a 
premultiplication of the matrix (multiplication on the Icft) by a non- 
singular matrix of a special form, as will readily be verified. 


Operation a) is equivalent to a premultiplication of the matrix by 
the matrix 


(32) « 


operation b’) is equivalent to a premultiplication by the matrix 
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(33) - ; 


] 
operation b”) is equivalent to a premultiplication by the matrix 


l 


(34) 


l 


Operations a), b’) and b”) are performed on columns by using just 
such elementary matrices! in postmultiplications of the matrix under- 


1 [Note that the elementary matrices may well be regarded as having been 
derived from the unit matrix by just such transformations, a), b’), and b’), as it 
is proposed to make upon the rows/columns of A. Thus, for example, the second 
elementary matrix of the example effects, by premultiplication, the following 
transformation of A: row 3+ row 2, and is itself the result of such an operation 
upon the unit matrix. By postmultiplication it effects: column 2+ column 3, 
and is this transformation of J,] 
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going transformation, (33) now effecting b”) upon columns, and 
(34) effecting b’). 
Examples of row operations: 


1 0 0 abe a 6 c 
0 0 x y z2 [=] ax ay az; 
00 1 u w “uoeovée=w 
1 0 O abe a b ¢ 
a | x y zi= x y z ; 
0 « u w Utax v+tay wW+az 
0 0 abe a b c 
0 a x y Z| =| xtau ytav z+aw 
0 1 “uv w u y w 


In future work we will often have to perform transformations of types 
a) and b’) upon matrices. The result of several row-transformations 
of this type is equivalent to the premultiplication of the matrix by 
some triangular matrix, i.e., one of the form 


(35) Y21 Ye2 --:- 0 


ese oe  #@ @  @  @  «e  @ 


Ynl Yn2 <+°* Yan 


with non-zero diagonal elements y;;.. Indeed, each separate trans- 
formation of form a) or b’) is equivalent to a premultiplication by a 
triangular matrix of the type indicated, and the product of two or 
more triangular matrices of like structure (i.e., both, say, dower 
triangular, as these) is again a like triangular matrix. 

It should be further noted that the result of several column- 
transformations of the form b’) and b’), such that to each, jth, 
column is added a multiple, m;;, of the elements of the ith column 
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(which itself remains unchanged), is equivalent to postmultiplica- 
tion of the given matrix by a matrix of the form: 


GOS, se, os ae Sor Book ae ek oc oe SO 
Oe De se ds es Sw Oe eo A BD 


(36) 


as es i es OF 
SOS et te Ek a ee 

11. Decomposition of matrices into the product of two triangular matrices. 
Triangular matrices, that is, matrices of the form 


C11 0 ... Q bi, bie ces b1, 
» «ne O Doo ... do 

(37) vel: Ee and ee less 
Cnt CpQ 2+ Oy 0 0 cms bon 


(no ¢;,=0, ho b;; =()) 


have a number of convenient properties. For instance, the deter- 
minant of a triangular matrix equals the product of the elements of 
the principal diagonal; the product of two triangular matrices of 
like structure is again a triangular matrix of the same structure; a 
non-singular triangular matrix Is easily inverted and its inverse is of 
like structure, etc. 

The following theorems are therefore of interest. 


THEOREM. On condition that the leading submatrices of the matrix 


Qi, &}12 «-- Qin 


&o1 a eee a 
A 22 2n 


ani 4.2 oee a 


are non-singular, t.e., that 


41, 19 
Q11 # 0, 


#0,..., [Al #0, 
@o, ao9 
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A may be represented as the product of a lower triangular matrix and an 
upper triangular matrix. 

The proof will be carried through by the method of mathematical 
induction. 

For n=1, the statement is obvious: (@),) = (4:;)(¢,,), and one of 
the factors may be taken arbitrarily. Let the theorem be true for a 
matrix of the (n—1)th order. We shall show it to be true for 
a matrix of the ath order. 

Partition the matrix A into a bordered matrix: 


a1) ai, 41n 
oe Qn} av, = A,-1 
an-1, n 
aay ann 4,1 ye a, n~l ann 


li 
( a | 
ote 
es a 
| 
a & 
P| 
3 
—eeEEE 


we shall seek a decomposition A =CB of the matrix A into the pro- 
duct of two matrices B and C of the required forms, first having 
partitioned these matrices into bordered form like that of 4: 


By the rule for multiplication of partitioned matrices: 


CB = i ° | iy I] 
x c¢ 0 6b 


RN ne 


= [Seite Go Day 
*éBe xy+c,,0 ‘ 


whence we have 
C,-1Ba-1 = Ay. 
Now such triangular matrices, C,_, and B,_4, exist, by the induction 


hypothesis. Furthermore, from the assumption that [4,_,|#0, it 
follows that |C,_,|/#0 and |8,_,]#0. 
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Now « and y are found by the formulas 
y = Cru, x = vBz.,, 


wherewith they are determined uniquely in terms of u and z. 
Thus it only remains for us to determine the diagonal elements 
€,, and ,, from the equation 


ConP nn =F Guy 4Y. 


The last equation shows that one of the diagonal elements may be 
taken arbitrarily. 

Thus the decomposition of a matrix into the product of two triangular 
matrices, lower and upper, will be unique if we prescribe values for the 
diagonal elements of one of the triangular matrices. 

It is convenient to consider, for example, that 6;,=1,1=1,..., 2. 
Then 


c 


an = Fan *Y 


and accordingly matrix C will be uniquely determined. 
12. Matrix notation for a system of linear equations. Let us consider 
the system of n linear equations in n unknowns: 


QyjX +Ayor%e+ °° > +4),x, = by 


Go1X1 +@ooxe+ +++ +@0.x, = b 
(38) 21*1 22*2 2nn 2 


G,1%1 Fa,9%o+ ++ * +£4,,x%, = 6. 


Utilizing the definition of matrix multiplication, the system may be 
written as a single equation in matrix notation: 


Q1, Gyn «es Ay *) by 
(38') @9, Gog «++ Gay *2 _ bo 
Q@n1 42 Ban Xn b,, 
or simply as 
(38") Ax = 3b, 


A signifying the matrix of the coefficients of the system, 5 the column 
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of constant terms, and x the column whose elements are the un- 
knowns. 

If the matrix of the system, A, is non-singular, we obtain at once 
the solution of system (38) by premultiplying (38°) by A-!: 

I 

39 x = Alb = — Bb, 
(39) Al 
B being the adjoint matrix of A. 

We shall show that the last formula is the matrix notation for the 
familiar Cramer’s Rule: 
(40) x, = aie 


A 


where 4; is the matrix that is obtained from A by replacing the ele- 
ments a,; of the ith column by the components 3; of 5. 
Indeed, the matrix equation (39) is equivalent to the 2 equations 


(40a) i ok Ay;6) + Ay,bo+ as +A .b 


; = gy (gz=1,..., n). 
Since the A,,; are the cofactors of the element a,; in the determinant 
of the matrix A, we obviously have 


(40b) Ay ;b; + Aojbo+ oe +A -b = |A,|, 


which proves our statement. 


§ 2. n-DIMENSIONAL VECTOR SPACE 


In what is to follow an important role will be played by the 
so-called n-dimensional vector space R,. A point X of such a space is 
an aggregate of 2 numbers, as a rule complex, arrayed in a definite 
order: 


(1) X= (x1, Royeeey X,)- 


X is also called an n-dimenstonal vector. The numbers x), x2, ...) %, 
are called the components of the vector. The number n is called the 
dimension of the space. 

Two vectors are said to be equal only if their corresponding com- 


ponents are equal. Fundamental operations on vectors are defined 
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as follows: If X¥=(x,, xo,...,x,) and Y=(y¥), yo,..., ¥,) are two 
n-dimensional vectors and a is an arbitrary complex number, we 
then put, by definition, 


(2) { X+Y = (xy +93, xo+Ye; eee y x, +Y,,) 


aX = (ax, dxX9,..., QX,). 


The addition of vectors satisfies the commutative and associative 
laws: 
X+Y¥ = Y4+X 
(X+Y)4+Z = X+(Y+Z). 


The addition of vectors is connected with multiplication by numbers 
by the distributive laws 


a(X+Y) = aX+aY 


(3) (a+b)X = aX+bX. 


The validity of all these laws follows directly from the definition 
of the operations. 

For vectors of an n-dimensional space a scalar product is intro- 
duced in accordance with the formula 


(4) (XY) = 3 ahh, 


where ¥, designates the complex conjugate of y,. 
It is readily verified that the scalar product has the following 
properties: 
1) (X,X) > OW X 40; (X,X) = On X = 0. 
2) (X,Y) = (¥, 2). 
3) (KX, +Xe, Y) = (X1, ¥Y)+(Xe, Y). 
4) (aX, Y) = a(X, Y). 
3) (4, Y¥y+ Ye) = (X, Yi) +(X, Yo). 
6) (X,aY) = a(X, Y). 
In addition, V(X, X) is called the length of the vector. In what 


follows we shall designate it by [|X]. 
Besides the n-dimensional complex space introduced above, it is 
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useful to consider also an n-dimensional rcal space, i.e., the aggregate 
of vectors with real components. 

In real space the scalar product is equal to the sum of the products 
of corresponding components of the vectors; the length of a vector 
equals the square root of the sum of the squares of its components. 

We shall most often have to deal with real n-dimensional space, 
turning to complex space only as occasion requires. 

1, Linear dependence. Avector Y=e,X,+eoXat+ +++ +6,,X,, 18 said 


to be a dinear combination of the vectors X,, Xo,... X,,. 
It is easily seen that if vectors Y,,..., Y, are linear combinations 
of the vectors X),..., X,,, any linear combination py, Y,+ +++ +,¥, 


will also be a linear combination of the vectors X,,..., X, 


me 


Vectors X), Xy,..., X,, are called linearly dependent if constants 


C1, €a,...,6,, exist, not all zero, such that the equation 
(5) 4X tegXot +--+ +6,X, = 0 


holds. If, however, this equatior. holds only when all the constants 
¢; are equal to zero, the vectors X,, Xo,..., X,, are said to be 
linearly independent. 

If the vectors X,,..., X,, are linearly dependent, then at least 
one of them will be a linear combination of the rest. For if, for 
example, ¢,,#%0, we find from (5): 


Cn-1 
Cn 


Xn-1 


(6) X, = -2X- +++ = 


THEOREM 1. [Jf the vectors Y,,..., Y, are linear combinations of 
the vectors X,,...,X,,, and k>m, the former set ts linearly dependent. 

The proof will be carried through by the method of mathematical 
induction. For m=1, the theorem is obvious. Let the theorem be 
true on the assumption that the number of combined vectors be 
m— 1. Under the condition of the theorem, then, 


Yy = Cy Xt) $0, Xn 


Ye = Cyt tise +e nX, 


Two cases are conceivable. 
1. All the coefficients ¢);,..., ¢ are equal to zero. Then 
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Y,,..., ¥, are in fact linear combinations of only the m—1 vectors 
Xo,...,X,, On the strength of the induction hypothesis, 
Y,,..., Y, will be linearly dependent. 

2. At least one coefficient of X, will be different from zero. 
Without violating the generality we may consider that ¢,,#0. 

Let us now consider the system of vectors 


Yi = Y,—2! ¥, 


~ ~  €yy 


yi= Y,-#! ¥,. 
“1 
The vectors thus constructed are obviously linear combinations 
of the vectors Xo,..., X,,,and the number of them is k—l>m-—-l. 
On the strength of the induction hypothesis they are linearly 
dependent, i.e., constants yg,..., y, that are not simultancously 
zero can be found such that 


yoYot +++ ty. = 0. 


Replacing Y5,..., ¥, by their expressions in terms of 
Y,,..., Y;, we obtain 


yiVityeYeot +: +y,¥y = 90, 


c. € 
where 7) = ee ye7 °°" a yp. The numbers y,..., yg are 


not simultaneously equal to zero and accordingly Y,..., Y, are 
linearly dependent. This proves Theorem |. 

A system of linearly independent vectors is said to constitute a 
basis for a space if any vector of the space is a linear combination of 
the vectors of the system. 

An example of a basis is the set of vectors 


(7) 


. @  @  @  @  @# oe @ 
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for it is obvious that for any vector X= (x, x2,..., x,) we have 
a = X16) + X2¢0 + ase + X,¢,- 


This we shall call the ital basis of the space. Such a basis 1s 
not the only one possille—quite the contrary: in the choice of a basis 
one may be arbitrary within wide limits. Despite this, the number 
of vectors forming a basis does not depend on its selection. In proof 
of this, let Y,,..., Y¥, and Z;,..., Z,, be two bases, and assume 
further thatk> m. The vectors Y,,..., Y, are linear combinations 
of the vectors Z,,..., Z,. In the light of Theorem ], Y,,...,¥; 
are linearly dependent, which contradicts the definition of basis. So 
k=m. Furthermore, since the initial basis is constituted by 2 
vectors, any other basis will also consist of n vectors. The number 
of vectors forming a basis thus coincides with the dimension of the 
space. 

Let U,,..., U, form the basis of a space. Any vector X will 
then be a linear combination of U,,... U;,: 


(8) NX = €,0,+F2.Un+ --- +€,U,. 
The coefficients of this resolution uniquely define the vector X, for 
if 
X= §,U,+--- +6U, = G0, +--+ +&U,, 
then (€,—&))U, + «++ +(&,—&)U, =0, and accordingly 
&,—§; =0,..., c= ¢, = 0, 


in view of the linear independence of the vectors Uj,..., U,,. 

The coefficients &,,..., &, are called the coordinates of the vector 
X with respect to the basis U,,..., U,. Note that the components 
of a vector x;,...,%, are the coordinates of the vector XY with 
respect to the initial basis. 

2. Orthogonal systems of vectors. The non-zero vectors of a space 
are said to be orthogonal if their scalar product equals zero. A 
system of vectors is said to be orthogonal if any two vectors of the 
system are orthogonal to one another. In speaking of an orthogonal 
system, we shall henceforth assume that all the vectors of this system 
are different from zero. 
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THEOREM 2. The vectors forming an orthogonal system are linearly 
independent. 
Proof. Let X;,..., X, be an orthogonal system, and let 


€,X) +60X0+ o8i8 +¢,X;, = 0. 
In view of the properties of the scalar product we have: 


O = (6X + +-- +6,X;, X;) 
€1(Xy, Xj) + +++ 0; (X, Xj) 4+ +++ ey ( XM, Xi) = ellXil?, 


Il 


and, since ||X,||?>0,¢;=0 for any :=1, 2,...,2. Thus the sole 
possible values for ¢),¢.,...,¢, in the equation ¢,X;+e.Xo 
+--+ +¢X,=0 are ¢)=tg=--- =¢,=0, ie., the vectors Xj, 
Xo,..., X, are linearly independent. It thence follows, first, that 
the number of vectors forming an orthogonal system does not exceed 
n, and, second, that any orthogonal system of » vectors forms a basis 
ofthe space. Such a basis is called orthogonal. If we have, in addi- 
tion, ||X;,||=1, the basis is said to be orthonormal. An example of an 
orthonormal basis is the initial basis. 

From any system of linearly independent vectors X,,..., 4, it 
is possible to go over to an orthogonal system of vectors X}, ...,X; 
by means of the process spoken of as orthogonalization. The following 
theorem describes this process. 


THEOREM 3. Let X,,...,X, be étnearly independent, An 
orthogonal system of vectors X\,..., X, may be constructed that ts con- 
nected with the original set by the relations: 


x1 =X, 
(9) Xo = Ketan X, 


X= My tay Ay os Hy pM. 


The proof will be by induction. 
Let X},..., X%_) be already constructed and different from zero. 
We seek X/ in the form 


(9’) Ka = XqtyXit 0+ +Ym-1Xn-1- 
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Choose the coefficients y1,..-, ¥m—-1 so that (X;,, Xj) =0 for 
j=l,...,m—1. This is easily done, for 


(Xi) X;) = (4, X;) +y;(X;, X5). 
Now (Xj, X}) #0, since X}#0 by the induction hypothesis, and it is 
accordingly sufficient to take 
= Sn XI) 
i> ORS) 
Replacing now Xj, ..., X;,_1 in equation (9') by their expressions 
in terms of X),..., X,,-1, we obtain finally 


Xm = Kg t mdi oo +n mim 


It remains to be proved that 73,40. But this is obvious, for 
otherwise the vector X,, would be a linear combination of the 
vectors },...,X,1, which contradicts the condition of the 
theorem. The basis of the induction exists, since for m=1 
the theorem is trivial. 

One may pass from any orthogonal system of vectors to the cor- 
responding orthonormal system by dividing each vector by its 
length. 

The process described permits of great latitude in the choice of an 
orthonormal basis, for one may pass from any basis to an ortho- 
normal one by orthogonalization and normalization. 

The scalar product of two vectors is very simply expressible in 
terms of the coordinates of these vectors with respect to any ortho- 
normal basis, for, if Uj, ..., U, is an orthonormal basis, and 


X= 6,0,;+---+6U,, Yon, +--+ +7,U, 


then 
(xX, Y) 


(Ci: t+ +++ +80, m0, + +++ +,U,) 
% 3 (GUs 10) = 3 > G(x Ui) = > ba 


t= t y= 


Thus the expression of the scalar product in terms of the co- 
ordinates of the vectors with respect to any orthonormal basis 
coincides with its expression in terms of the components of the 


vectors, i.c., in terms of the coordinates with respect to the initial 
basis. 
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3. Transformation of coordinates. Let us elucidate the change in the 
coordinates of a vector that accompanies a change of basis. 
Let ¢1, ¢g ..., é, and eé}, €9, ... é, be two bases, and let 


€) = 411€) tagyégt+ +++ +4,)6, 


€ = Gyoli tdgolot+ ++ +4, 26, 


(10) 


, 


en 


44,01 +Ao,ee+ °+* Tange, 


We connect with the transformation of coordinates a matrix A, 
the columns of which consist of the coordinates of the vectors 
é}, €2,.... €, With respect to the basis é), é2,... é,, i.€., the matrix 


411 Aye eee Ay 


Go, @oy «.- Go, 


(11) A=|° lt |e 


Qi) 4qn oe np 


The matrix A is non-singular, for it has an inverse, by means of 
which the vectors ¢), éo, ..., €, are expressible in terms of the vectors 
ae Corer 

Now designate by x;,..., x, the coordinates of a vector X with 
respect to the basis ¢,,¢,..., €,, and by xj, xj,...,%, its co- 
ordinates with respect to the basis ¢}, ¢j,...,¢,. Let us determine 
the relation of dependence between the old and the new coordinates. 
We have: 

X = xe; +Xgeot ++ > Hx 6, = X4Cptxeea+ oo + Xnee 
=X (41 1¢) +ao1ee+ +++ +4,1¢,) 
+ Xn (4101 +4g0¢2+ +++ +4,2¢,) 
-4- r e r) e ° . ry ° e e e e ° 
+x) (21,€1 tGo¢ot °°° +4,,¢,) 
= (84)X] tH ayoxet +++ Hay yXn)e 
+ (491X4 + door + ++ + + daqgXn)es 
+ e e e . ° ° e e ° e e e 
+ (2,31 +a 2x2 + ean + An Xnyens 
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whence, on the strength of the linear independence of the vectors 


C15 Coy « « oy &: 


Kp = AX Ht ayaxgt 6° $44 yXn 


Xg = Gq)Xy+goxgt +++ +agrn 


(12) 


x, = 4,\*1 + a,oX0 + ene + 4,,Xn- 


The last equations may be written in matrix form. Of course 
the set of a vector’s coordinates can be considered either as a column 
or asarow. In view of the definition of matrix multiplication, we 
can postmultiply a square matrix only by a column, not by a row. 
In future we shall, therefore (except by special stipulation), take 
the coordinates of a vector as a column. Often in arguments where 
the basis is fixed (for instance, when the vector is given in terms of 
its coordinates with respect to the initial basis), we shall ideritify the 
vector with the column of its coordinates. 

Equation (12) may be written in the form 


(13) x= Ax’, 
where 
x) x] 
Xo Xo 
(14) xa] and x’ = 
x, Xn 


are the coordinate columns of the vector X with respect to the bases 
€},...,¢, and ¢},..., é, respectively. 

4. Subspaces. A set of vectors XCR, such that any linear com- 
bination of the vectors of this set is itself'a vector of the same set, is 
said to be a subspace of the space R,. Ifa group of vectors U,,..., U,, 
that are linearly independent—or even linearly dependent—be 
given, then the set of all possible linear combinations of them will 
obviously constitute a subspace. A subspace constructible in this 
manner is said to be the subspace spanned by the system of vectors 
Sopra 5 ae 

We shall show that a basis exists in every subspace, i.e., that 
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there is a set of linearly independent vectors, by lincar combinations 
of which one may exhaust the entire subspace. Let us construct 
the basis in ‘the following manner. Take first an arbitrary vector 
X),, different from zero, and consider all its linear combinations, i.c., 
all vectors of the form ¢cX,. If these exhaust the entire subspace, 
A, then forms its basis. Ifthe contrary is true, a vector X. will be 
found, linearly independent of X,. Consider the set of lincar 
combinations of X, and Xz: if they do not exhaust the subspace, a 
vector will be found linearly independent of them, and so forth. 
The process cannot go on interminably, for in the space A, there 
cannot be more than 2 linearly independent vectors. We shall thus 
have constructed a finite system of vectors X),..., X, such that 
their linear combinations exhaust our subspace, i.e., we shall have 
constructed our basis. 

It will be remarked that the reasoning set forth indicates much 
latitude in choice of a basis. However the number of vectors forming 
a basis will not depend on the manner of its selection, in the light of 
Theorem 1, That number is called the dimension of the subspace. 

Note that the set composed solely of the null vector, as also the 
set composed of the entire space, will each be subspaces in the sense 
of our definition. We shall regard them as trivial subspaces. 

5. The connection between the dimension of a subspace and the rank of a 
matrix. We introduce the important concept of rank, appropriate 
to any rectangular matrix A, 


A = Q04 oo ee Go, 
ey e2 Any 


Any determinant whose rows and columns “‘fit’? the rows and 
columns of a matrix is called a minor of this matrix. More exactly, 
a minor of order k of the matrix A is a determinant of the Ath order 
formed from the elements situated at the intersections of any & rows 
and any k columns of the matrix A, in their natural arrangeinent. 

The order of whatever non-vanishing minor is of largest order is 
called the rank of the matrix A. In other words, the rank of a 
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matrix is a number r such that among the minors of the matrix 
there cxists a non-zero minor of order r, but all minors of order 
r+1 and higher are equal to zero or are not composible (as in the 
case, for instance, of a rectangular m xr matrix, m>r). 

The following important theorem is valid: 


THEOREM. The maximum number of linearly independent rows of a 
matrix, as also the maximum number of linearly independent columns, coin- 
cides with the rank of the matrix. 

From this theorem it follows directly that the dimension of the 
subspace spanned by the vectors U,,..., U,, equals the rank of the 
matrix composed of the components of these vectors. 

Indeed, if the rank of a matrix whose columns are the components 
of the vectors U;,..., U,, equals 7, then of these m vectors r will be 
linearly independent, and these will correspond to the linearly 
independent columns of the matrix; all the rest of the columns will 
be linear combinations of them. Any vector subspace is a linear 
combination of the vectors U,,..., U,,, which are themselves linear 
combinations of but r selected linearly independent vectors. Con- 
sequently any vector is a linear combination of r vectors, and there- 
fore the rank r of the matrix in question coincides with the dimension 
of the subspace. 


§3. LINEAR TRANSFORMATIONS 


I. Let us associate with each vector X of a space a certain 
vector Y of the same space. Such an association we shall call a 
transformation of the space. We shall designate the result of the 
application of transformation A to the vector X by AX. 

We shall call the transformation A linear if 


1. A(aX) = aAX, for any complex number a; 
2. A(X, +X2) = AX, + AXs. 


We shall define, furthermore, operations upon linear transforma- 
tions. The product of the linear transformations A and B, AB=C, 
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will be a transformation constituted by the transformations B and A 
in turn, B being completed first, and then A. 

The product of linear transformations is a linear transformation, 
as is readily seen, since 


C(X, + Xo) = A(B(X,+X2)) = A(BX, + BX2) 


(1) = ABX,+ ABX, CX,+CXz, 


CaX = ABaX = AacBX = cABX = «CX. 


The sum of the linear transformations A and B will be a trans- 
formation C which associates the vector X with the vector AX + BX. 
This sum of linear transformations is obviously itself a linear trans- 
formation. 

2. Representation of a linear transformation by a matrix. Let us 
choose, in the space &,, some basis ¢), ¢o,...,¢, A linear trans- 
formation relates to the vectors of the basis the vectors Ae, 
Aés,..., Aeé,. 

Let Ae,,..., Ae, be given in terms of their coordinates with 
respect to the basis ¢,, éo,... €,; 1.€., let 


Ae, = 411; + @q1€o+ eas + 4,16, 
Aéz = Qy9@)+Go2¢o+ +++ + Ant 


(2) 


Consider the matrix A, its columns composed of the coordinates 
of the vectors Ae,, Aéo,..., Ae,: 


@i1 412 a1, 
Qo oo ao 
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We shall show that the matrix A uniquely defines the linear 
transformation.! 

Indeed, if the matrix A is known for the linear transformation, 
ic., if Ae), Aes, ... Ae, are determined, this is sufficient to find the 
transformation of any vector, for if 


X= xe; + °°: +X, 
then 
AX = x,Ae,;+ +--+: +, Ae,. 


Hence the coordinates of the transformed vector are easily found, 
for we have 


Y=AX= 5 446, = S x,Ae; = ) D agsxsees 
kal j =) ka] 


whence 
Rn 
| 2; Agi X55 
(= 
or, in matrix notation, 
(4) y = Ax, 


where y and x are columns of the coordinates of vectors Y and X. 
Conversely, an arbitrary matrix A may be connected with a cer- 
tain linear transformation. Indeed, the transformation given by 
the formula 
y = Ax, 


where y and «x are, as above, the columns of coordinates of the 
vectors Y and X, is linear for any matrix A. 

The established one-to-one correspondence between transforma- 
tions and matrices is preserved when operations are performed upon 
transformations, for the matrix of the sum of transformations equals the 
sum of the matrices of the summand transformations, and the matrix of a 
product of transformations equals the product of the matrices corresponding 
to the factor transformations. 

3. The connection between the matrices of a linear transformation with 
respect to different bases. We will now elucidate how the matrix of a 
linear transformation changes with a change of the basis of the space. 


1 Note, however, that the matrix of the coefficients in the relations (2) forms a 
matrix which is the transpose of that that we connect with the linear transformation. 
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Assume that from the basis ¢,, ..., ¢, we have passed to the basis 
€i,- ++, €,, and let 


€y = Cy:0) Heyyeot +++ $6,146, 
€y = Cyots Heyvtet +++ +e,0€, 


C= Cyl tlogeot +++ +6,,€ 


The coordinates of any vector of the space will have been changed 
accordingly by the formula 


e=Cy’, 
where 
C1. C12 £1, x) x) 
Coy Cao Co Xo Xo 
CG pe _ ’ x“ = ’ x’ = 
Cal nat = Onn xn Xn 


The matrix of the transition, C, is evidently non-singular. [t will 
coincide with the matrix of the linear transformation sending the 
basis ¢}, éy,..., &, into the basis ¢}, ¢o, . . . e. 

Let us now consider a linear transformation A, and let the matrix 
A correspond to it with respect to the basis e,, es, ...,¢,, and the 
matrix B with respect to the basis e}, ea, ... 5 €n- 

If x is the column of the coordinates of the vector X with respect 
to the basis e,,.., ¢,, and x’ that with respect to the basis ej, ... , ¢,, 
y and y’ being the analogous columns for vector Y, we have 


y = Ax 
y’ = Bx’. 
But x=Cx’, y=Cy’, and therefore 
Cy’ = AC’ 
or 


y’ = Bx’ = C-1ACx’, 


Thus similar matrices correspond to the same linear transforma- 
tion with respect to different bases. Furthermore, the matrix by 
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means of which the similarity transformation is effected coincides 
with the matrix of transformation of coordinates. 

4. The transfer rule for a matrix in a scalar product. Let X and Y be 
two vectors given by their components with respect to the initial 
basis: X=(x),..-,%,)) Y=(¥i,--+5¥,), and let A be a linear 
transformation with matrix A=(a,,).. Designate by A* the linear 
transformation with matrix A*, the clements of which are the 
complex-conjugates of their counterparts in A, and which are placed 
in transposed positions: A* =(a,,)* =(a;,') =(a,;). We shall prove 
the following formula: 


(AX, Y) = (xX, A*Y). 
In demonstration, we have 


3 3 dug = (% A*Y). 


(AX, Y) = 2, 2 X49; 


5. The rank of a linear transformation. Let A be a certain linear 
transformation. The set of vectors AX will obviously constitute a 
subspace, which we shall denote by AR,. 

The dimension of this subspace is said to be the rank of the érans- 
formation A, 

We shall show the rank of a transformation to be equal to the 
rank of the matrix corresponding to this transformation on any 
basis whatever, ¢), ¢:,...,¢, Obviously the subspace AR, is 
spanned by the vectors Ae,, Aes,..., Ae,. The dimension of AR, 
is accordingly equal to the rank of a matrix whose columns are com- 
posed of the coordinates of the vectors Ae,, Aev,..., Ae,, i.€., to 
the rank of a matrix corresponding to the transformation. 

Since the dimension of a subspace does not depend upon the 
selection of the basis, it follows from the foregoing that the ranks of 
similar matrices are equal. 

6. The proper vectors of a linear transformation. By a proper vector 
(characteristic vector, latent vector or etgenvecior) of a linear transformation 
A is meant any non-zero vector X such that 


(6) AX = 2X, 


where / is any complex number. 
The number A is called a proper number (characteristic number, latent 
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rool, or eigenvalue) of the transformation. The spectrum of the trans- 
formation is the aggregate of its proper numbers. 

The proper numbers and proper vectors of the transformation 
may be determined in the following manner. Let the transforma- 
tion A be connected with the matrix A= (a;,) with respect to some 
basis; let the coordinates of the proper vector X, with respect to this 
basis, be Xp 2 0 2 Xe 

The coordinates of the vector AX will then be: 


| >: Giprgy ces 2 aus | p) 
k=] k=1 


and thus for the determination of x, x9,...,4%, and the proper 
number A we will have the system of equations: 


441)X%1 + AyoXet +++ +4),%, = Ax, 


4g 1X1 + dg0X%o + rae + ao,*, = Axe 


(7) 2 
43%) tA,or%n+ °++ +4,,x, = Ax, 

or 

(7') Qa )X) + (doy —A)Xo+ a +49,%, — 0 


By1X1 + 4,9%o + as + (4,,—4)%, = 0. 


This system of homogeneous equations in x,,..., x, will have a 
non-zero solution only in case 


4;,—A Qi ooe Gj, 

49) Qog—A e+e Go, = 0 
= 3 

a1 a,0 a Pr | 


i.e., if A is a zero of the characteristic polynomial of the matrix. 
Thus the following is valid: 
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THEOREM. The proper numbers of a transformation coincide with 
the zeros of the characteristic polynomial of a matrix that is connected with 
this transformation with respect to an arbitrary basis. 

From the theorem known as the fundamental theorem of higher 
algebra, we know that every polynomial has at least one zero. A 
linear transformation will consequently have at least one proper 
number, which may be complex even though the matrix of the trans- 
formation be real. In view of the theory of linear homogeneous 
systems of equations, there will be a non-zero solution of system (7) 
for each proper number, i.e., with each proper number at least one 
proper vector is associated. 

Obviously if X is a proper vector of the transformation A, then, 
for all ¢ 0, ¢X will also be a proper vector of transformation A cor- 
responding to the same proper number. Furthermore if,several 
proper vectors correspond to some one proper number, then any 
linear combination of them will be a proper vector of the trans- 
formation associated with the same number. The set of proper 
vectors corresponding to the same proper number forms a linear 
subspace. We shall establish that its dimension, /, does not exceed 


the multiplicity of the proper number. Indeed, let X,,..., X; be 
linearly independent proper vectors corresponding to the same 
proper number A,. Construct a basis of the space X),..., X,, 


having taken as the first / vectors the vectors X;,..., X;. With 
respect to this basis the linear transformation under consideration is 
connected with a matrix whose first / columns have the form 


a, 0 ... 0 

0 A 0 
(7’b) 

0 0 a, 

0 0... 0, 


for AX, =A,X,,..., AX,;=A,X; Now (A—A,)! is a factor of the 
characteristic polynomial of this matrix, and accordingly 4, is of 
multiplicity & not less than J, i.e., /<k. It would naturally be sup- 
posed that /=4, i.c., that to a k-multiple root of the characteristic 
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polynomial there correspond & linearly independent proper vectors. 
But this is in fact not true. In reality, the number of linearly inde- 
pendent vectors may be less than the multiplicity of the proper 
number. 

Let us confirm the preceding statement with an example. Con- 
sider the linear transformation with the matrix 


[34] 


Then |A — Al] =(A—3)2, and thus 4=3 is a double root of the char- 
acteristic polynomial. 

The system of equations for determining the coordinates of the 
proper vector of the transformation A will be: 


3x) +X = 3x) 
3x. = 3x. 


whence x, =0, and thus all the proper vectors of the transformation 
in question wil] be (x,, 0)=x,(1, 0). So in this instance only one 
linearly independent vector is associated with a double root. 

Generally speaking, the coordinates of a proper vector on the 
chosen basis are to be determined from the system (7) of linear 
equations, in which for 4 the proper number A, is substituted. But 
as is known from the theory of systems of linear equations, the num- 
ber of linearly independent solutions of a homogeneous system equals 
n—r, where r is the rank of the matrix composed of the coefficients 
of the system. Therefore if r denotes the rank of the matrix 4— AJ, 
l=n—r. Thus n—rgik, and the equality does not always hold.! 

In case the basis does not change in the course of the argument, 
we shall often identify the linear transformation with the matrix of 
the linear transformation with respect to this basis, and any vector 
of the space with the column of its coordinates. 

On this agreement, it makes sense to speak of a proper vector of a 
matrix, understanding by this a column <x satisfying the condition 


Ax = he. 


We remark that if a proper number of a real matrix is complex, 
1V. 1. Smirnov, {1}; A. G. Kurosh, [1)}. 
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the coordinates of an associated proper vector will be complex. A 
vector whose coordinates are the complex conjugates of those of a 
given proper vector of a real matrix is also a proper vector of that 
matrix, and is associated with the complex conjugate proper num- 
ber. To convince oneself of this, it is enough to change all numbers 
in the equation Ax = Ax into their complex conjugates. 

7. Properties of the proper numbers and proper vectors of a matrix. We 
shall establish several properties of the proper numbers and proper 
vectors of a real matrix. 

First of all we note that a matrix and its transpose have identical 
characteristic polynomials and consequently identical spectra. 
This is evident since |A’—AJ|=|A—AlI|, on the strength of the fact 
that a determinant is not altered when its rows and columns are 
interchanged. 

Now let A, and A, denote distinct proper numbers of the real 
matrix A, and 4, the complex conjugate of 4,. As we saw above, 
A, is also a proper number of matrix A, and thus also of the trans- 
posed matrix A’. Let xX, be the proper vector of the matrix A 
belonging to the proper number 4,, and X! the proper vector of the 
matrix A’ belonging to the proper number /,.. We shall show that 
X, and X! are orthogonal. 

With this object in view, let us form the scalar product (A.X,, X') 
and reckon it by two methods. 

By one method we have 


(AX,, x5) = (A,X, X}) = A(X,, X'). 


On the other hand, since matrix A is real, we have A* =A’, and 
therefore 


(AX,, X') = (X,, A'Xs) = (X,, 4,X%) = A(X, X'). 
Thus A,(X,, X$) =A,(X,, X{). But the condition was that 1,#4,, and 


s 
therefore (X,, X{) =O, which is what was required to be proved. 
In case all the proper numbers are distinct, the demonstrated 
property gives n*—n relations of orthogonality between the proper 
vectors of matrices A and A’. We shall later return to these 
properties in more detail. 


For a real symmetric matrix, the properties of orthogonality are 
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considerably simplified, thanks to the fact that all ts proper numbers 
are real. In proof, letting A and X be respectively proper number 
and vector, we have (AX, X)=A(X, X); (AX, X)=(X, A’X) 
=(X, AX) =(X, AX) =i(X, X). Thus (A—(/)(X, X)=0, and as 
(X, X)>0, A=A, ie., A is real. 

From the reality of the proper numbers of a real symmetric 
matrix it follows that vectors with real components may be taken as 
the proper vectors belonging to those roots; the components will 
indeed be found by solving the linear homogencous system with 
real coefficients. 

The orthogonality property of the proper vectors of a real sym- 
metric matrix is very simply formulated in view of the coincidence 
of the matrix with its transpose and the reality of the proper num- 
bers, viz.: proper vectors belonging to distinct proper numbers are orthogonal. 

8. The proper numbers of a positive-definile quadratic form. A homo- 
geneous polynomial of the second degree in several variables 
%1,++.+,%, is called a quadratic form. We shall consider only those 
with real coefficients. Any quadratic form may be written as 


n 
P(x), Xo,...5%,) = 2 By Xj Xj 


where 4;, = 4,;. 
A quadratic form is said to be postitve-definite if its values are 
positive for any real values of x,,... , x,, not all zero simultaneously. 
It is evident that the diagonal coefficients of a positive-definite 
form are positive, for 


a;, = O(1,0,..., 0), a». = @(0, 1,...,0),... 
a,, = (0, 0,..., 1). 


Denoting by X the vector with components (x;,...,4%,), We 
may write a quadratic form as (AX, X), A being the matrix com- 
posed of the coefficients of the form. This matrix is symmetric, on 
the strength of the definition. The proper numbers of the matrix 
are called the proper numbers of the quadratic form. In view of 
the previous results, all proper numbers of a quadratic form are real. 

We shall show that if a quadratic form is posttive-defintte, ils proper 
numbers are positive. 
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In demonstration, let XY be a real proper vector, belonging to A, a 
proper number of the matrix of the form. Then, since the form 1s 
positive-definite, (AX, X)>0. On the other hand, (AX, X) 
=A(X, X). Thus 

1 = AX) 
(x, X) 
But both numerator and denominator of this fraction are positive, 
and consequently A>0, which is what was required to be proved. 

Let there now be given any real, non-singular matrix A. Ob- 
viously B=A’A is a symmetric matrix, since B’=(A‘A)’=A‘A’ 
=A'A=B. 

We shall show that a quadratic form with matrix B is positive- 
definite. We have, indeed, 


(BX, X) = (A'AX, X) = (AX, AX) >0 


for any real vector X. 
We shall establish, lastly, that if A is the matrix of a positive- 
definite quadratic form, (AX, X)>0 even for a complex vector X. 
In proof, let X= Y+1Z, where Y and Z are vectors with real 
components. Then 


(AX, X) = (AY+iAZ, Y+iZ) 
= (AY, Y) +i(AZ, Y)—i(AY, Z)+(AZ, Z) 
= (AY, Y)+(AZ, Z)>0 


because (AZ, Y) =(Z, AY) =(AY, Z). 
In complex space, instead of the quadratic form one deals with an 
Hermitian form, an expression of the type 


TMs 


Bj pXiX ps 


under the condition that a,,=4,,. 

The matrix of an Hermitian form is called Hermitian (or Hermitian 
symmetric) ; a linear transformation with an Hermitian matrix relative 
to an orthonormal! base is called self-conjugate. It is obvious that 


Dd Ft; % = (AX, X). 
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To show that all the values of an Hermitian form are real, we 
have only to note that 


(AX, X) = (X, A*X) = (AX, X). 


If all the values of an Hermitian form are positive, it is called 
posilive-definite. 

It can be shown that the proper numbers of an Hermitian matrix are 
real. The proper numbers of a positive-definite Hermitian form are positive. 

9. The reduction of a matrix to diagonal form. Let us consider the 
matrix A all of whose proper numbers, 4,,..., A,, are distinct, and 
the transformation A connected with it with respect to the initial 
basis. It will have n distinct proper vectors X,,..., X,. We shall 
show that the vectors X,,..., X, are linearly independent. 

Assume the contrary: let the vectors X,,...,X, be linearly 
dependent. Without detriment to the generality we may assume 
that the vectors X,..., X,, where k <n, are linearly independent, 
and thus that the vectors X,4,,..., X, are linear combinations of 
them. In particular, let 


(8) X,= 5 6X; 


then 


On the other hand, 


k 
AX. = Aakn = > Awe: Xj) 

ist 
whence 


(9) 3 Adak: = 0. 


But A,4#A,;, on assumption. Thus, since the vectors X; are linearly 
independent, all the coefficients ¢; equal zero, the therefore X, =0, 
which contradicts the definition of a proper vector. So the vectors 
X;, Xo,..., X, are linearly independent. Let us adopt them as a 
new basis of the space. With respect to the new basis the linear 
transformation A will be connected with a matrix whose columns 
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are composed of the coordinates of the vectors AX;, AXo, ..., AX, 
with respect to the basis Xj, Yo,..., X,- 
But 
AX, = A,X, 


and the matrix of the transformation on the new basis will conse- 
quently be diagonal: ' 4), do, ... A, 7 

So the linear transformation A has, with respect to the initial 
basis, the matrix A, and with respect to the basis of the proper 


vectors, the diagonal matrix A), Ao, ... a, _y Accordingly, on the 
strength of what has been noted above, 
(10) V-IAV =" Ay, dese ey An p 


where V is the matrix whose columns are the coordinates (with 
respect to the initial basis) of the proper vectors. 

Observation. If the proper numbers of a matrix are of multi- 
plicity greater than one, but to each proper number there correspond 
as many proper vectors as it has multiplicity, the matrix may also be 
reduced to diagonal form. This will be the case, for example, with 
symmetric matrices: it can be proved that to each proper number of a 
symmetric matrix there correspond as many linearly independent proper vectors 
as the multiplicity of the proper number. Moreover, the linearly inde- 
pendent proper vectors belonging to a single proper number may be 
subjected to the orthogonalizing process. We have secn, too, that 
the proper vectors of a symmetric matrix that belong to distinct 
proper numbers are mutually orthogonal. Thus for a symmetric 
matrix it is possible to construct an orthogonal system of proper 
vectors forming a basis for the whole space. 

The question of the transformation of a symmetric matrix to 
diagonal form is closely connected with the theory of quadratic 
forms. 

10. The proper numbers and proper vectors of similar matrices. It has 
been established that similar matrices have identical characteristic 
polynomials, and consequently identical spectra of proper numbers. 

Wc have explained the geometrical cause of this circumstance, 
viz.: similar matrices may be regarded as matrices of one and the 
same transformation, referred to different bases. Therefore the 
proper vectors of similar matrices are the columns of the coordinates 
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of the proper vectors of the transformation under consideration, 
with respect to different bases, and are thus connected by the 
relation x’=C-!x, C being the matrix of transformation of co- 
ordinates. This circumstance may be verified formally: if Ax = Ax, 
(C~1AC) (C-tx) = A(C-1x), 

11. The proper numbers of a polynomial in a matrix. Let A be a 
matrix with proper numbers 4;,...,4,, and let (x) =a9-+a x 
+--+ +4," be the given polynomial. Then the proper numbers 
of the matrix g(A) will be p(A;), (42), ..., p(A,). 

This is readily established for a matrix all of whose proper numbers 
are distinct. Indeed, such a matrix can be reduced to diagonal 
form by a similarity transformation: 

Pg oe Oe Ly ey Poe A, C. 
Accordingly, 
g(A) = C-lg! 2), Ao, --., A, C 


9 ae | ¢ 
But 


pT Aas s+ Ay) = M(Aa)s Aa)» = + = PCAn)_p 


which follows from the fact that 


VAisaeg Aa SP Meee Ae 
Consequently 


pay... A, ) = =u A ates 


i 2 k Mo: ak 
DX aAj,..-, > ada. 
k=O jo 


“p(Ar)s @(Aa)s + «= PCr), 


Thus the matrix (A) is similar to the matrix" g(A}), . . . .p(A,) 4 
and accordingly its proper numbers are ¢(A;), p{Ao),.--, @(A,)s 
Q.E.D. : 

This result remains true for any matrix, of which one may readily 
convince oneself, for example, by considerations of continuity. 

We particularly note that the proper numbers of the matrix Aé 
are A‘. 

12. The normalization of the proper vectors of a matrix. The second 
group of orthogonality relations. Let A be a real matrix whose proper 
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numbers, A), Ao,..., A,, are distinct, and let X,, Xo,..., X, be the 
proper vectors corresponding to them. As we saw, the transposed 
matrix, A’, has the same proper numbers. Let X}, X9,..., X%, be 
the proper vectors of the matrix A’, and their enumeration be so 
chosen that X; and X; belong to complex-conjugate proper numbers. 
We established above that the following orthogonality relation 
holds: (X;, X;)=0 for 14y. We shall show now that, having chosen 
X31, 49,-++, Xn In any manner (they are determined but for a 
numerical multiplier), we may normalize the vectors X), Xo,..., X_ 
so that (Xj, X;)=1. 

In demonstration of this, the vectors X), Xo, ..., X, are known to 
be linearly independent, and they accordingly form a basis of the 
space. Resolve X; in terms of this basis: 


Ki = iA tyeket ++ t+ yXs. 
Forming the scalar product (X;, X‘), we obtain 
(X;; X;) = yi(X;;, Xi)+ ++ + (Xj, Ay) et +y,(X} X,,) 


= yi(X;, ¥)), 
whence we conclude that (X;, X;) =a;40, for (X{, X;) >0. 
Adopting instead of the vectors X;,...,X, the vectors 


] ] ; ‘ atae : 
— X,..., a X,) we arrive at the required normalization, since 
a1 a 


, 1 ] 
(x1, = Xi) = =X) = 1 
From the relations of orthogonality and normality set forth above, 
we may derive another group of relations between the components 
of the proper vectors of the matrix A and its transpose. 
Form the matrices 


4 e | 

iy MQ, 0 Xp od) ed | © 

X= *12 X22 +++ Mag ae ae *21 *ae Xeon 
é ‘ C 

*in Xon eee Xan Xn x2 eee Xnn 


The columns of the matrix X are composed of the components of 
the vectors X,,...,X,. The rov's of the matrix X¥’ are composed 
of the numbers complex-conjugate with the components of the 
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vectors X},...,4,- (We observe that the numbers that are the 
complex conjugates of the components of the vectors X},..., X, 
are the components of the vectors x; ..., 42, which will also be 
proper vectors of the matrix A’ belonging to the proper numbers 
Ai,...+,A,- Thus the ith row of the matrix X’ and the ith column 
of the matrix X are composed of the components of proper vectors 
of matrices A’ and A, belonging to the same proper number A,;, and 
not to proper numbers that are complex conjugates of cach other.?) 
It is readily seen that: 


(11) X’X = 1, 

for we have the element of the ith row and jth column of the matrix 

X'X equalling > x1x,,=(X,X1)=64, 
f= 


delta: 


p where 6;; is Kronecker’s 
5. 8 tA 
yd, dss. 


Thus X’ and X are mutually inverse matrices, and accordingly XX’ 
is likewise equal to /. This gives a second group of orthogonality 
and normality relations between the proper vectors of the matrices 
A and A’, viz.: 
(12) xi1%}1 +x;oXj9+ aii +X: 5% jn =0 (t#)) 
XpyXp1bXjperjet +++ HX;_gXZ_ = I. 

1 [This may perhaps be clarified by a summary in matrix notation. By the 

definitions at the beginning of Paragraph 12, 
’ 
A'X( = AX}. (2) 


From (2), 
A’X{ = 1Xj, (2a) 
i.e., A'R( = A,Xj, for Aisreal. (2b) 
Now by definition of the matrix .X’, 
Xi 
en ee 
R; 


From which, with (2b) and (1) in view, the author’s statement is confirmed.] 
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Thus for a matrix of the second order the ordinary conditions of 
orthogonality may be written in the form (we preserve only one 
index of the components of the proper vectors, designating the first 
components by x, the second by y): 

*x2+y1¥2 = 0 

oxi + yey = 0, 
and the normality conditions: 


xy +914, = | 
XoXo + YoYo 


The new relations will be 


= 


l 
1. 


MYitxeys, =O xx] +%9x2 

yixityers =O yi tY¥2¥2 

Observation. In case the proper numbers of the matrix are 
multiple, but to each proper number there correspond as many 


linearly independent proper vectors as the multiplicity of the root, 
these indicated properties of the proper vectors hold as before. 


§4. THE JORDAN CANONICAL FORM 


We have proved above that if a matrix has n distinct proper 
numbers, it may be brought into diagonal form by a similarity trans- 
formation. Given the presence of multiple roots, however, such a 
transformation may not always be possible. Nonetheless the problem 
of reducing the matrix to as simple a form as possible by a similarity 
transformation can well be posed. The problem is equivalent to 
discovering a basis with respect to which the linear transformation 
connected with the given matrix would have a matrix of simplest 
form, and the latter proves to be the Jordan canonical form. 

Proof of a fundamental theorem to the effect that any matrix may 
be brought into the Jordan canonical form by a similarity trans- 
formation is rather complicated and we will not dwell on it here.! 
We shall limit ourselves to a description of this canonical form. 

1 [See Frazer, R. A., Duncan, W. J., ano Corrar, A. R., [1], § 3.16 for the 


gist of the proof, or, for the proof proper, Turnputt, H. W., AnD Arrxen, A. C., 
[1], Chapters V-VI.] 
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A matrix of the following form is called a canonical box: 


Se | 


On its principal diagonal the single number A; is everywhere to be 
found; directly under the diagonal (in the subdiagonal) are disposed 
elements that are all units; all the rest of the elements are zero. 

A canonical box cannot be simplified by utilizing a similarity 
transformation. It is obvious that a canonical box has the sole 
multiple proper number /;. It may be easily verified that a 
canonical box has only one proper vector. ‘The minimum poly- 
nomial of a box coincides with its characteristic polynomial, viz., 
it equals (A; — A)" where m; is the order of the box. 

The Jordan (classical) canonical form is a quasi-diagonal matrix 
composed of canonical boxes: 


40 0 0 
1 a, 0 0 
Oo 1 A, 0 
0 0 0 re 


—-— ~~ © 
~ CO SO 
oO Oo O& 
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It is admissible that the same number 4; appear in several canonical 
boxes. 

All the numbers 4; figuring in the different boxes are proper 
numbers of the canonical matrix, and the multiplicity of a proper 
number equals the sum of the orders of the boxes in which it figures 
as diagonal element. In proof of this, by the theorem concerning 
the determinant of a quasi-diagonal matrix, the characteristic poly- 
nomial of the canonical matrix equals the product of the character- 
istic polynomials of the separate boxes, each of them equal to 
(A; — A), where A; is the proper number and m, is the order of the 
tth box. Hence follows directly the statement under proof. 

The determination of the Jordan boxes for a given matrix 
A presents certain difficulties. | The characteristic polynomial 
I¥(A;-—A)™ coincides with the characteristic polynomial of the 
original matrix, and it is consequently possible to find it without 
knowing the canonical matrix itself. Nonetheless, a knowledge of 
the characteristic polynomial still does not make possible the com- 
plete determination of the canonical form, for a proper number A; 
of multiplicity k; there may correspond several Jordan boxes con- 
taining this number as a diagonal element, and regarding them only 
the sum of their orders will be known, not the order of each box in 
particular. Ifthe canonical form is to be fully determined, a know- 
ledge of the “elementary divisors’ of the matrix must be drawn 
upon. 

Designate by D;(A) the greatest common divisor of all the minors of 
the :th order of the determinant |A—AJ|. In particular, D,(A) 
coincides with the characteristic polynomial. It can be proved that 
all D;(A), as D,(A), are common to the class of similar matrices. It 
can be proved, moreover, that D,;_,(A) divides D,(A)}. 

Put 


Dj{A) 
D;_1(A) 


= E;(A); 
obviously 


D,(a) = [T] E,(A). 


1 [See, e.g., TURNBULL, H. W., AND ArrKen, A. C., [1], p. 23 ©] 
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D,(A) 


It turns out, moreover, that £,(A) =D. (a) 
n~l 


is the minimum poly- 


nomial of the matrix. 
Resolve £;(A) into linear factors. Then 


Ea) = IT (A; —a)ow. 
ie 


Rn 
Here s denotes the number of distinct proper numbers, > m;;=k 


. > 
ira] 


$s n 
2, 2, m;,;=n. It is obvious that among the exponents m,; only 
jelic 


some will be different from zero. 

The binomials (A;—4)™y are known as the elementary divisors of 
the matrix 4. A knowledge of the elementary divisors permits us 
to construct the canonical form, viz.: the Jordan boxes are con- 
structed by starting from the numbers 4;, and the orders of these 
boxes are equal to the exponents m;;. The number of boxes con- 
taining 4, equals the number of exponents m;; not equal to zero. 

In case the elementary divisors are linear, 1.e., if all the non-zero 
exponents m;; are equal to one, the Jordan boxes degenerate into 
diagonal elements, and the canonical form turns out to be simply a 
diagonal form, wherein, of course, a single proper number will 
appear as often as a diagonal clement as it has multiplicity as a zcro 
of the characteristic polynomial. 

The converse is also obvious, for it is clear that ifa matrix can be 
brought to diagonal form, its elementary divisors are linear. ‘{here- 
fore matrices with distinct proper numbers, as also symmetric matrices, have 
linear elementary divisors. 

Ifall the elementary divisors (A; — A)" are relatively prime (which 
occurs only in case D,_,(A)=1), each proper number appears in 
only one canonical box, and the order of the box equals the multi- 
plicity of the corresponding proper number. Only in this case does 
the minimum polynomial coincide with the characteristic poly- 
nomial, 

Let us now consider the matrix transforming a given matrix A into 
canonical form. With this object in view we introduce into the 
discussion the linear transformation connected with the matrix A 
with respect to the initial basis. Then the columns of the trans- 
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forming matrix will be components of the vectors of that basis in 
terms of which the linear transformation in question is describable 
as a canonical matrix. 

Let this canonical matrix have the form 


A, 
As 
Pr 
where 
A. “Or ae 20 
l 0 
A, = A, 
0 0 ... A 


and the order of A, equals m,; let USY, UD, ..., UM, ren ci 


.. +, US) be the corresponding basis. Then the following formulas 
for the transformation hold: 


AUT = 4,U7)+ UD 
AUD = 4,U2+U3 


AUY, = 24,U9 4U0 


Mtr —} itp 
( ) (r) 
AU® = 1,U% 


for allr=l,...,5. 

We see that among the vectors of the canonical basis are the 
proper vectors of the given matrix, one per box. It can be proved 
that with this all linearly independent proper vectors of the matrix 
are exhausted, and consequently the number of linearly inde- 
pendent proper vectors of the given matrix equals the number of 
canonical boxes in its canonical form. In particular, the number 
of linearly independent proper vectors belonging to a given proper 
number equals the number of canonical boxes containing this 
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number. It is not of greater multiplicity than the proper number, 
and is equal to this multiplicity in case, and only in case, all boxes 
containing the given proper number are of order 1, i.e., when the 
corresponding elementary divisors are linear. 


§5. THE CONCEPT OF LIMIT FOR 
VECTORS AND MATRICES 


Let a sequence of vectors X"™, X,...,.X%,... with com- 
ponents (x!),..., x ),..., (x@),..., x),... be given. If a 
limit exists for each component: lim x) =x,, the vector .¥, with 


k—>a0 
components x;,...,,, is called the i:mit of the sequence X™, 
X(2),..., X@),... and the sequence itself is said to be convergent to 


the vector X. This is written in the form X‘)—YX or lim X® =X. 
k=» 


In the same fashion, given a sequence of square matrices A‘), 
A®),,.., A®,... with elements (af), (af?)),..., (aff), ...,the 
matrix A with elements a;; = lim af? is called the limit of the 

koe 


sequence, if all these limits exist. 
In accordance with such a definition of a limit, an infinite series of 
vectors X04 X(2)4., 6.4 Xb) +. «+ is said to be convergent if lim (X™ 


k-—»@® 


+ X(2)4 6... 4X) exists; this limit is called the sum of the given 
series. Obviously it is necessary and sufficient for the convergence 
of a series of vectors that all the series composed of their correspond- 
ing components, i.e., components bearing the same indices, con- 
verge; the sums of these series are the components of the sum of the 
series of vectors. 

The concept of the convergence of a series of matrices is defined 
analogously. 

In applied questions it is usually important to judge not only the 
convergence of a sequence or series, but also to judge the rate of 
this convergence. With this object in view, the introduction of the 
norms of vectors and matrices is quite useful. A norm may be 
introduced in different ways, and in different cases one or other 
norm will prove to be most convenient. 
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Generally, the norm of a vector X is an associated non-negative 
number |/.X|| satisfying the following requirements: 
1) ||X|| > 0 for X ¥ 0 and |{0| = 0; 
2) [eX]! = |e| ||X]| for any numerical multiplier ¢; 


3) |JX+ Yl < ||X||+||¥|| (the “triangular inequality’’). 
From requirements 2) and 3) it is readily deduced that 
|X— YI) > | ux-H¥i|. 
Indeed, we have 


XI] = |X - V+ Fi < |X YF 


and therefore 
|X— ¥|] > |XI]—-||¥ I. 
But 
|X— Fil = |Y—XI] > VIIA. 
Consequently 


Wx —¥y > [XUN | 


We shall henceforth make use of the following three ways of 
assigning a norm: if X= (x), xo, ~~~. 5%); 


I. |X], = max |z,| 
HT. WX py = [ail + |x] + +++ +], 
TT. |X Uay = V [xy]? + [xol2+ - +> +[x, 12. 


It is obvious that for all three norms all the requirements |)—3) 
are fulfilled. 

The concept of the norm of a vector generalizes the concept of the 
length of a vector, since for length all the requirements |)—3) are 
fulfilled. The third norm introduced by us is indeed none other 
than the length of the vector. 

Furthermore, it is easily established that a necessary and sufficient 
condition that the sequence of vectors X‘#) converge to the vector X 
is that ||.X‘) — X||-0 for each of the three norms indicated. For the 
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first norm this is obvious. For the second and third norm this 
follows from the obvious inequalities 


Xl < Xl < alXh 
WX < UX < VallXlh- 


It is easily shown that for convergence of a sequence of vectors 
X‘) to a vector X it is necessary and sufficient that ||X —.X||>0, 
whatever norm satisfying conditions 1)—3) we may choose. Here, 


if L®—>X, |X) |[->|| XI], for | AAO] | <A — £0. 


In an analogous fashion, the norm of a square matrix A is a non- 
negative number ||d|| satisfying the conditions 


1) |jAl] > O1ifA ¢ O and |O|| = 0; 
2) [lea] = lel IIAll; 

3) ||A+ Bi] < [Al] +B); 

4) ||ABll < |All |B). 


Just as in the case of the norms of vectors, the condition ||A“) — Al]->0 
is necessary and sufficient in order that A‘4)—A, and just as in the 
case of the norms of vectors, it follows from A™—>-A that ||A‘:j|->|| A]. 
The norm of a matrix may be introduced in an infinite variety of 
ways. Because in the majority of problems connected with csti- 
mates both matrices and vectors appear simultaneously in the 
reasoning, it is convenient to introduce the norm of a matrix in such 
a way that it will be rationally connected with the vector norms 
employed in the argument in hand. We shall say that the norm 
of a matrix is compatible with a given norm of vectors if for any 
matrix A and any vector X the following inequality is satisfied: 


AX] < [All Al. 


We will now indicate a device making it possible to construct the 
matrix norm so as to render it compatible with a given vector norm, 
to wit: we shall adopt for the norm of the matrix A the maximum 
of the norms of the vectors AX on the assumption that the vector ¥ 
runs over the set of all vectors whose norm equals unity: 


I[4]| = max ||44|l. 
WX st 
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In consequence of the continuity of a norm, for each matrix A 
this maximum is attainable, i.c., a vector X) can be found such that 
|Xoll = 1 and ||AXol|=|:4l]. 

We shall prove that a norm constructed in such a manner satisfies 
requirements 1)—4), set previously, and the compatibility condition. 

Let us begin with the verification of the first requirement. 

Let A#0. Then a vector X, ||X|/=1, can be found such that 
AX #0, and accordingly {|AX||#0. Therefore |{Al|| = max [|4X || 40. 

UN i= 


If, however, A= 0, ||Al| = max |'0.||/=0. 
Xo 


Second requirement. On the strength of the definition, |{cAl 
= max ||cAX|]. Obviously |{cAX]| = |e| ||A.X|| and thus 


[|All gat to le| AX] = |e] max |]4X]] = |e] [4]. 


Let us verify, furthermore, the compatibility condition. 


Let Y#0 be any vector; then X = Y will satisfy the condition 


that ||X|[=1. Consequently 
AY] = WACHYUADI = IPT AAT < tY ITA. 


Third requirement. For the matrix A+8 find a vector Xo such 
that ||4+ Bl] =|!(4 + B)Xol] and ||Xo]|/=1. Then 


||A + Bll = |(A +B) Xo 
= ||AXo+ BXoll < ||AXol]+l|BXoll < |All ||Xoll +118! [Xoll 
= ||All+||8]. 


Lastly, the fourth requirement. For the matrix AB find a vector Xo 
such that ||Xpo||=1 and ||ABXo||=||ABl|. Then 


|ABl| = ||ABXol| 
= ||4(BXo)Il_ < All |BXoll < WALI Fl | Xoll 
= |[Al] 18]. 


We have verified the satisfaction of all four requirements and the 
compatibility condition. A matrix norm constructed in this manner 
we shall speak of as subordinate to the given norm of vectors. It is 
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obvious that for any matrix norm, subordinate to whatsoever 
vector norm, ||/||=1. 

Let us now construct matrix norms subordinate to the three 
norms of vectors introduced above. 


L XI, = max |s;. 
8 
The matrix norm subordinate to this vector norm is 
n 
All; = max > |a,,|- 
é ket 
In proof, let ||X||=1. Then 
R n Rn 
|AX]] = max | 2, a;,%,| < max 2, la;s| [x,| < max 2 Ia;,|. 
t = a Oe) 8 = 


Consequently 

max ||AX|| < max 3 |a;4|. 

DX t= i kel 
We shall now prove that Lg AX] is in fact equal to max > la;,l. 
For this we shall construct a vector Xo such that [|[.X%9|=1 and 
||AXoll = max >> la;,|. Letting > la;,| attain its greatest valuc for 
i=j, and then taking as the component x{° of the vector Xo: 
ap = Kut, if a,40, and x=1 if a,=0, we have, obviously, 


} 
||Xol|=1. Furthermore, 


% R n . : 
2 art?) < 2 lanl < 2 lau] fori ¥s 
ko) kel kal 


and 


R 
(0) 
> A 4x 
| ee 


Rn 
= 3| a;,\. 
k=l 


Consequently 


fi n 
max = > |4¢n| = max 5 | 4, |. 
k= 1 i ons 


= a; xy 
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Thus |AXol| = max 2 a,,|, Q.E.D. 
IT. ln = > [x;|.- 
The matrix norm subordinate to this vector norm is 
lly = max > lau. 
In proof thereof, let |'‘X||=1!, then 


waxy = 3 |S aun 


Rn 8 
< > D lace [ral 
i=] ko] 
R Rn 
< > Is, | | 3 ull 
k=) = 
a R n 
< [ max 2 ul] 2 [| < max 2 layl- 
k t=] ke] gk ge] 


A 
Now let us take a vector Xo of the following form: let > |a,,| 
iol 
attain its greatest value for the column numbered j. Put x =0 
for k#j and x =1. Obviously a vector constructed in this manner 
has its norm equal to unity. Furthermore 


AX] = Dd | D> ax] = 2 la;,| = max > |a,,|- 
tof} | kw) iol kK kev} 
Thus 
max ||AXoll = max > lai, 
iat 
Q.E.D. 
IT. WXllin = Ra |x,|2 = (X, X). 


The matrix norm subordinate to this vector norm is 


Allis = VA, 


where A, is the largest proper number of the matrix A’A. In proof, 
we have 


All = max ||AX]]; 
NX l=24 
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but 
WAX, = (AX, AX) = (X, A’AX). 


The matrix A’A is symmetric. Let 24,24. 2 -+-+ 24, be its proper 
numbers and X, X2,..., X, be the orthonormal system of proper 
vectors belonging to these proper numbers. 

Now take any vector X with its norm equal to unity and resolve 
it in terms of the proper vectors: - 


X= 65X+6oXo+ cee +¢ Xx. 


Then 
(X,X) = &+84+--- +0 =1. 
Moreover, 


[JAX |]2 = (X, A’AX) 
= (6X, t+ +++ +6,X,, AX t+ +++ +6,4,%,) 
= Ait +--+ +A < Ay(cf+ +--+ - +0?) = Ad. 
For the vector X=.X;: 


JAXQ\|? = (X1, AAX,) = (X], AX) = Ay. 
Thus 


max ||AX|| = V/A, 
re) ED WX! 


We shall now prove several theorems connected with the concept 
of limit. 


THEOREM 1. In order that A»—>0, it is necessary and sufficient 
that all the proper numbers of the matrix A be of modulus less than unity. 

Proof. Assume for simplicity that the matrix A can be brought 
into diagonal form: A=CAC-1, where A=! Aj, Ao,..., 4, _ and 
A, 4o,...,4, are the proper numbers of matrix A. Then 


An=CAC-1, It is obvious that An=! 9", ya iy 
that A=—>0, it is necessary and sufficient that A™~0, for which it is 
in turn necessary and sufficient that all proper numbers 4, Ao, ..., A, 
have modulus less than unity. 

In case the matrix A cannot be brought into diagonal form, the 


theorem is proved cither with the aid of considerations of con- 


In order 
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tinuity or by passing to the Jordan canonical form. We shall not 
dwell on the details of this proof. 

The conditions given in Theorem } are inconvenient for testing, 
inasmuch as they require foreknowledge of the proper numbers of 
the matrix A. We shall therefore establish some simpler sufficient 
conditions rendering lim A™=0. 


n—>» a 


THEOREM 2. Jn order that An—>0, tt is sufficient that any one of 
the norms of A be less than unity. 

Proof. On the strength of the fourth requirement of a norm, we 
have 


ile, < Jam] [All < [ld] Lali? < +++ < [lel 
Therefore ||A™||->0 if |[A]|<1, and thus, in view of the foregoing, 
An—>0. 


Combining Theorems | and 2, we arrive at the following result: 


THEOREM 3. No proper number of a matrix exceeds any of its 
norms in modulus. 


Proof. Let |]4|]}=a@. Consider a matrix B= A, where é is 


any positive number. We have 


a 
ate 


BI] = < |, 


and accordingly B=->0 as m—>oo. On the strength of Theorem 1 
its proper numbers have modulus less than unity. But it is obvious 


that the proper numbers of the matrix B equal 


oar A;, where A; are 


the proper numbers of the matrix 4. Thus st <I, i.e., [Aj] <ate. 


Since € may be taken arbitrarily small, |A;| <a. 


THEOREM 4. dn order that the series 
(1) TeAG cae GuAwa aoe 


converge, tt is necessary and sufficient that A~->O as m—>oao. In such a case 
the sum of series (1) equals (I—A)7}. 
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Proof. The necessity of this condition is obvious. We shall 
show that it is sufficient. 

On the strength of Theorem 1, all proper numbers of the matrix A 
are less than of unit modulus. 

Accordingly 

\7—A| 4 0 

and therefore ({—A)—! exists. 

Consider the identity 


(1+A+A2+ --+ +Aé)(F—A) = I— Abt, 
Postmultiplying it by (J—A)~!, we obtain 
[+A+A2+ +--+ +Ab = (1—A)—-!— Abti (J —A)7), 
whence it follows that, as k—>oo, 
I+A+-++ +Abt>({—A)-} 
since Akt1—»0, 


Thus 
I[+A+ +++ +Ak+ «++ =({—A)=}, 


which is what was required to be proved. 

In the light of Theorem 1, the necessary and sufficient condition 
for the convergence of the series (1) is the inequality |A,;| <1 for all 
proper numbers of the matrix A. A sufficient criterion of con- 
vergence, in view of Theorem 2, is the inequality ||Al] < 1, whatever 
one of the norms be employed. Given that this condition is satisfied, 
it is easy to give the following estimate of the rate of convergence of 
the series (1): 


THEOREM 5. /f |\|Alj< I, 
|(Z-A)-97- (I+ A+ +++ +4*)I] < 
Proof. We have: 
(I—A)-1—(I4 A+ +++ +Ab) = Abt + Abt24 ... 


[Alt 
1 — |All 


whence 


k+1 
\(I—A)-1-— (TE A4 +++ +A8)l| < PANEER oe oS ; 


and the theorem is proved. 


CHAPTER II 


SYSTEMS OF LINEAR EQUATIONS 


This chapter is devoted to three problems that are closely 
related to each other: the problem of solving a non-homogeneous 
linear algebraic system, the problem of inverting a matrix, and the 
problem spoken of as elimination. 

In theory all of these problems are soluble simply enough. How- 
ever, if a matrix of high order is connected with the problem, the 
actual solution requires a great number of computational operations. 

Numerical methods solving the stated problems are divisible into 
two groups: exact and iferative methods. By exact methods we under- 
stand methods that give the solution of a problem by means of a 
finite number of elementary arithmetic operations. The number 
of computational operations necessary for the solution of the problem 
depends only upon the form of the computational scheme and upon 
the order of the matrix defining the given problem. Inexactitude 
in the solution found occurs as the result of the inevitable rounding- 
off of the figures in the course of the computation. Along with this, 
one may run up against the phenomenon of the disappearance of 
significant figures in the course of the computation, as the result of 
the subtraction of two numbers differing little from each other. This 
loss of significant figures may occasion such an important reduction 
in the accuracy of the result that it is often necessary to alter the 
computational scheme because of it, or re-do the work with a greater 
number of significant figures in the intermediate calculations. 
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The fundamental method of this group is the method based on 
the idea of elimination. ‘The algorithm of this method, with which 
the name of Gauss is associated, consists—when applied to the solu- 
tion of a non-homogeneous linear system—of a chain of successive 
eliminations by means of which the given system is transformed into 
a system with a triangular matrix whose solution presents no 
difficulty. 

Many different computational schemes have been developed and 
are currently employed for all three of the problems mentioned. 

Those schemes known as compact schemes are outstanding among 


these. The compact schemes are based on the possibility of con- 
n 


2 445; 
ducting the calculations of expressions of the form —— all at one 


time on the modern calculating machine, dispensing with the 
recording of the results of the intermediate computations of the 
separate summands a,b,; this type of computation is spoken of as 
accumulation. By accumulative computation the rounding error ts 
naturally reduced sharply and the volume of intermediate results 
to be recorded is considerably diminished. 

A method founded on the idea of bordering also belongs to the 
exact methods. At the root of this method lies the representation 
of a given matrix in the form of matrices situated one within the 
next, starting with a matrix of the first order. 

Iterative methods afford a means by which a system of linear equa- 
tions may be solved approximately. The solution of a system by 
iterative methods is obtained as the limit of successive approxima- 
tions computed by some uniform process. The convergence of 
these approximations depends essentially on the elements of the 
matrix defining the given problem. The rate of the convergence 
depends also on a happy choice of the initial approximation on 
which the iterative process is founded. 

In particular, for the basic iterative processes, there exist matrices 
for which a process converges only slowly, or even diverges. 

The immense advantage of the iterative schemes consists in the 
simplicity and uniformity of the operations to be effected, and there- 
fore in the possibility of completely mechanizing the process of 
computation. 
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In the continuation of this chapter, we shall assume that all the 
given numbers defining the condition of the problem are real. 


§ 6. GAUSS’S METHOD 


In this section we shall recall to the reader the well-known 
algorithm for solving a system of equations by a successive elimina- 
tion of the unknowns. As has already been mentioned above, it is 
associated with the name of Gauss. For simplicity of reasoning and 
in order to carry the calculation to the end, we shall consider a 
system of four equations in four unknowns. 

Thus let there be given the system of equations 


411%] + Ay o%X%2+419%3 + Ay 4X4 = 415 


4gX1 + AgoXy + AggXq + doqX4 = Aes 


(t) 


€31X1 + Ag0Xo + 453X%4 + AgqXq = O35 
44) Xy + AgoXa + Aq9%o t+ AyqXy = Ags 


Let us divide the coefficients of the first of equations (1) by the 
coefficient a,,, which we shall speak of as the leading element, and 
use the designation 


= Ol, , 
(2) bi; nae j > l. 


We will have obtained a new equation 
(3) %y + by 0X2 +51 3x3 + Oy yxy = O45. 


Let us now pass on to an auxiliary system, composed of the three 
equations in three unknowns that are obtained as the result of elimi- 
nating x, from the last three equations of (1) by means of equation 
(3). This elimination is easily done by multiplying equation (3) by 
Gs}, 43), 44, in turn (i.e., by the leading elements of the second, third 
and fourth rows) and subtracting it from the corresponding equa- 
tions of system (1). The coefficients of the new system obtained as 
the result of the elimination of one variable, we shall designate by 
Qa: .3: 


(4) Fe | = @;;— 4; 15),; thy 2 2; 
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Proceeding, we shall now divide the coefficients of the first equa- 
tion of the new system by the “leading” element ago. ;. 
We shall have obtained the equation 


(5) X_ + bo3.1%9 + bog yXq = bo6-3, 

where 

(6) bos. = i; j> 2. 
22°1 


Continuing the process analogously, we shall pass to a system of two 
equations in two unknowns, whose coefficients have the form 


(7) Qij-2 = A j-3—Sjonbo51, 19 2 3. 


Dividing the coefficients of the first equation of this system by the 
leading element agg.. and defining 


(8) bsg = U2; jf > 3, 
833-2 
we shall obtain the equation 


(9) X3 + b3q.0x4 = b35-0. 


Finally, a last step will lead us to a single equation containing a 
single unknown with coefficient a4,.3. Upon dividing this equation 
by Qg4-q, WE obtain 


¥4 = 545-3. 


Collecting all equations with coefficients 4;;.;_1, j>1, we shall have 
obtained a triangular system equivalent to the given onc; its solution 
will be the solution of our system. We note that the process 
described is possible only on condition that none of the leading ele- 
ments by which we have, in the course of the work, divided the 
coefficients of the first equations of the intermediate systems, are 
equal to zero. 

So for solving the given system we first construct an auxiliary 
triangular system, and then solve it. The process of finding the 


TABLE I. 


Single-Division Scheme 


poyyayy s,ssnvy 


*a ™s 2 x 
Ors a4 ay a 1.00 | 0.42 0.54 O66 |) |i 2.92 
as ax a eas 0.42 1,00 0.32 0.44 0.5 2.68 
ay ang a5 ay 0.54 0.32 1.00 0.22 0.7 2.78 
2a3 ea ay aa 0.66 0.44 0.22 1.00 0.9 3.22 
bis bra bis Bee 1 0.42 0.54 0.66 0.3 2.92 
an Pee ee ee ~~ 0,82360 0.09320 0.16280 0.37400 1.45360 
a33.3 Gene 239.4 436-4 0.09320 0.70840 ~- 0.13640 0.53800 1.20320 
a eee ee biti 0.16280 | -0.15640 0.56440 0.70200 1.29280 
bles beak Bais Bick t 0.11316 0.19767 0.45410 1.76493 
ate; Nl cuea 7 wae 1b as - 0.69785. =| «0.15482 | 0.49568 1.03871 
Ges | Geas Gans fess —0.15482 0.53222 0.62607 1.00547 
3 Deas Baus baes 1 — 0.22185 0.71030 148852 
ees erm ar eran ier ee 0.49787 0.73804 1.23591 
1 a sl { 1.48240 2.48240 
1 Xs y 1 1.03917 2.03916" 
Xs Es i 0.04348 1.04348 
Pa Pa 1 ~ 1.25780 0.25979) 


Horizontal sums in superior position. 


L9 
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coefficients of the triangular system we shall call the forward course, 
and the process of obtaining its solution, the return course, or back 
solution. 

The scheme that we have described we shall call the single-diviston 
scheme (see Table I). 

We shall say a few words concerning a control (check) that it is 
expedient to employ when computing by the single-division scheme. 
This control is based on the following circumstance. If in the 
system given we makc the substitution €,=x,;+ 1, then for determin- 
ing the x, we will obtain a system with the former coefficients, and 
with its constant side equal to the sums of the elements of the rows 
of the matrix of the coefficients (plus the constant terms). There- 
fore having formed the sums of the elements of each row (the con- 
trol sums), we shall perform upon them the very same operations as 
upon the rest of the clements of the row. In the absence of com- 
putational blunders, the numbers found must coincide with the 
analogous sums of the transformed rows. The return course is 
controlled by finding the numbers £, and their coincidence with the 
numbers x; + 1. 

We will briefly explain Table I. 

The forward course is carried through as follows. Having copied 
the matrix of the coefficients (including the constant terms and the 
control sums), we divide the first row by the leading element and 
copy the result as the last row of the matrix. Next we compute the 
elements a;;.;; 1,j 22, of the first auxiliary matrix: taking any elec- 
ment of the given matrix, we subtract from it the product of the 
leading element of the row to which it belongs by the last element of 
the column to which it belongs. Continuing, the process is re- 
peated. The computation of the forward course is concluded when 
we arrive at a matrix consisting of one row. 

In the return course, one utilizes the rows containing units, begin- 
ning with the last: in the last of these rows, in the column of constant 
terms, we obtain the value of the last unknown, and in the control 
column, the control value. Continuing, the values of the unknowns 
are obtained sequentially, as the result of subtracting from the 
elements of the next-to-the-last column the products of the cor- 
responding 6-coefficients by the values of the unknowns found pre- 
viously. The unit symbols displayed at the end of the scheme aid in 


Taste II. 


The Single-Division Scheme: Symmetric Case 


0.54 
0.09320 
0.70840 


0.11316 
0.69785 


0.66 
0.44 
0.22 
1.00 
0.66 


0.16280 


— 0.13640 
0.56440 
0.19767 


0.53222 


— 0.22185 


0.49787 
] 


0.3 
0.5 
0.7 
0.9 
0.3 


0.37400 


0.53800 
0.70200 
0.45410 


0.49568 


0.62807 
0.71030 


0.73804 


1.48240 
1.03917 
0.04348 
— 1.25780 


1.45360 
1.20320 
1.29280 
1.76493 
1.03871 
1.00547 
1.48844 
1.23591 
2.48240 
2.03916 
1.04348 
— 0.25779 
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locating the coefficients that correspond to the given x in the neccs- 
sary rows. Thus 


(9b) xq = bo5-1 — bog-1%4 — b93.4%3 
= 0.45410 —0.19767 - 1.48240 —0.11316- 1.03917 = 0.04348. 


In conclusion we note that the operation of subtraction may be 
replaced by addition if in copying a row with coefficients 6, one 
reverses their signs. 

The number of multiplications and divisions necessary for finding 
the solution of a system of n equations by the single-division scheme 


is equal to q ("2+ 3n— 1). 


In case the matrix of the coefficients of the system is symmetric, 
l.e€., a;;=a,;;,, we have, obviously, 4;;.,=4,;;.,. We may therefore 
omit writing the elements below the principal diagonal. The 
single-division scheme, as modified for symmetric matrix, is shown 
in Table IT. 

The leading element of the row, omitted in writing the coefficients, 
(and which we need for the computation of the elements of the 
auxiliary matrices) is easily found as the uppermost element of the 
column containing the diagonal element of the given row. The 
control column contains, as before, the sums of all the elements of 
each row, even including those omitted in tabulation. 

In case several equations with the same matrix must be solved, 
their solutions are naturally sought simultaneously, the constant 
terms being entered in adjacent columns. The control sum is 
formed as the sum of the elements of the rows of the expanded 
matrix. The scheme for solution of several equations is given in 
Table ITT. 

Other computational schemes as well are possible for Gauss’s 
method. We mention a scheme of multiplication and subtraction 
in which the coefficients of the auxiliary systems, which we shall 
designate by 4;,.,, i, >, are obtained as the difference of the 
cross-products 


Ajj a Aggy Ajju-i —Aggyy Appar 


With this scheme the forward course requires no divisions at all; 
they are all moved over into the return course. 


Tasce III. Single-Division Scheme: Several Equations 


Xe Xy Xs 
0.42 0.54 0.66 0.25 : 0.15 
1.00 0.32 0.44 0.45 F 0.30 
0.32 1.00 0.22 0.65 0.45 
0.44 0.22 1.00 0.85 : 0.60 
0.42 0.54 0.66 0.25 0.15 


! 


a 


9.82360 | 0.093920 | 0.16280 | 0.34500 0.37400 | 0.23700 2.03560 
0.09320 0.70840 — 0.13640 0.51500 0.53800 0.36900 2.08720 
0.16280 — 0.13640 0.56440 0.68500 0.70200 0.50100 2.47880 
1 0.11316 0.19767 0.41889 0.45410 0.28776 2.47159 


0.69785 — 0.15482 0.47596 0.49568 0.34218 ages 
— 0,15482 0.53222 0.61680 0.62807 0.45415 2.07643 
— 0.22185 0.68204 0.71030 0.49033 2.66082 


poyjayy S$ SSnvy 


(6) 
0.49787 0.72239 0.73804 0.53006 peas 


J 1.45096 1.48240 1.06466 4.99805 


(3) 
1.00394 1.03917 0.72652 3.76964 


0.01847 0.04348 ~- 0.00490 1.05 Wy: 
= 1.25752 — 1.25780 — 0.94294 - 2.45828 


Horizontal sums in superior position. ~J 
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The method of pivotal condensation (dominant elements) is quite 
generally encountered. In this scheme the elimination is con- 
ducted as follows. Select the largest element of the matrix -1, a,,, 
which is called the pivot (dominant element), and compute the 
multiplier 

m, = sell 
a; 
for all :4k. Now multiply the pivot row (i.c., the Ath row) of the 
matrix A by the multipliers m; in turn, and add the result to the 
corresponding ith row. As a result one obtains a new matrix of 
which the jth column consists of zeros. Discarding this column and 
the Ath row, we go over to the first auxiliary matrix with a number 
of rows less by one. The subsequent eliminations are conducted 
each time by means of the pivotal element of the transformed matrix. 
The sought-for triangular system is obtained by collecting the pivot 
rows. 

In concluding this section we remark that in computing the 
coefficients of the intermediate systems by the single-division scheme 
we may encounter a disappearance of significant figures. If this 
occurs for a leading element this implies an important loss of 
accuracy. 

In such a case the pivotal condensation scheme may be con- 
veniently used to carry the computations through. 

In such modification, the process is always applicable, provided 
the determinant of the system is different from zero. 


§7. THE EVALUATION OF DETERMINANTS 


Gauss’s method, developed in the preceding section for the 
solution of a linear system, also may be applied to the computation 
of determinants. We shall dwell separately on a description of the 
corresponding scheme, since the computation of determinants is 
frequently met with in applications. Let 


ai a\o2 eee Gin 


ao Gao eee a: 
A 21 4 2n : 
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and lect a,;40. Remove the element a), from the first row. We 
then have, using the symbols of § 6, 


ant ane eee a 


RR 


Next take from each row the first row multiplied by the first clement 
of the particular row in hand. We will obviously obtain 


| bye eee b1, Qoo+4 eee Ban] 
0 Aun ao 
pL . acl 
A = 4), = 41 
O ayo ees) Aan A,0-) Gant 


With the determinant of the (n—1)th order that remains, we shall 
now deal just as before, provided aus., 49. 

Carrying through the process, we shall find that the sought-for 
determinant is equal to the product of the leading clements: 


a = 4))490-1--.- Aanen—le 


If at any step it should turn out that a,;.;_;=0 or a;;.;_1 is close 
to zero (which would imply the reduction of the accuracy of the 
computations), the order of the rows and columns of the determinant 
could be rearranged beforehand so that a non-vanishing element 
appears in the upper left corner. 

The best result—in the sense of reliability—will be obtained if at 
each step of the process the maximum element (in point of absolute 
valuc) of the matrix whose determinant is being computed is brought 
to the upper left corner. 

The number of multiplications and divisions necessary for the 
computation of a determinant of the nth order is equal to 
n—| 


(n®+n+3). 


In Table IV is given the scheme for computing the determinant 
of the example. 


TaBLe IV. Computation of a Determinant 


0.42 
1.00 
0.32 
0.44 
0.42 
0.82360 
0.09320 
0.16280 
] 


0.54 
0.32 
1.00 
0.22 
0.54 
0.09320 
0.70840 

— 0.13640 
0.11316 
0.69785 

— 0.15482 
! 


0.69785 


0.16280 
— 0.13640 
0.56440 
0.19767 


0.15482 
0.53222 
— 0.22185 
0.49787 
0.49787 


1.07960 
0.66520 
0.59080 
1.31083 
0.54303 
0.37740 
0.77815 
0.49787 


$L 
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From an inspection of the process of computing the determinant 
we see that, with the exception of the last multiplication, it coin- 
cides with the forward course of the Gauss process, applied to a 
system with the matrix for which the determinant is computed. 

The familiar rule of Cramer (§ 1) shows that the solution of a 
linear system may be found in the form x= i=1,...,, where 
AA denotes the determinant of the coefficients of the system, and 4; 
the determinant of the coefficients in which the :th column has been 
replaced by the constant terms. Thus for the solution of the 
system one must compute n+! determinants of the nth order. 

Comparing the Gauss process for the solution of a system with 
that for computing a determinant, we see that the volume of com- 
putations for the solution of a system only slightly exceeds that for a 
single determinant. The use of Cramer’s formula for the numerical 
solution of a system is thus inefficient. Essentially in the Gauss 
method the computation of all the determinants 4 and 4, is effected 
simultaneously, the division by 4 =4 1490.1 . . - @nan-,-) Deing accom- 
plished gradually, one factor per step. 


§8. COMPACT ARRANGEMENTS FOR THE 
SOLUTION OF NON-HOMOGENEOUS 
LINEAR SYSTEMS 


In §6 we saw that the solution of a system of linear equations 
by the single-division scheme reduced to the determination of the 
coefficients a;;., of auxiliary systems (including the constant terms), 
and the coefficients 5;,;.;_, of the equations of a final triangular 
system. Only the coefficients 4;;.;_; were necessary to us there for 
obtaining the solution of the given system, and the numbers q;;., 
played an auxiliary role, being necessary only to determine the num- 
bers 5; ;.;.,, which we shall indeed henceforth denote simply by 3 
dropping the index denoting the number of the step. 

We shal] show! that the numbers 6;; can be obtained by a process 
of accumulation permitting one to dispense with the computation 
and recording of all the coefficients a;;.,. 


1 P, Dwyer, [1]. 
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Let us single out the clements of the first column of each auxiliary 
matrix, 4;;.;-1, 72J, designating them ¢;;, 12). 

Analysing the process of computing the coefficients of the 
auxiliary matrices, we see that 


ijk = Oijee — Giga Ogg a = Fyj-g—1— Caley 


5 5-4-9 — Sin 10 4-1; — S084; = ore 
(1) k 
= A;;— 65161; — Cabos — ++ + —Cjgbyy = Oj, — p> C4154; 
Thus any element a;;., is expressible in terms of an accumulation of 
the elements we have singled out, ¢;;, and the numbers 4; ;. 

In particular, for the elements ¢;;,12j, and 6;;, 1<j, themselves, 
the recurrence formulas 


iy? 


j-1 
Cp = App = a 2, Citi 12) 


(2) 


hold. The constant terms of the transformed system are obviously 
to be determined in accordance with these same formulas. 

The return course remains the same as in the full-length singic- 
division scheme. 

Computation by the compact scheme may be conveniently 
arranged as is shown in Table V. 

Here we effect the computation of the elements ¢ and 8 succcs- 
sively anglewisc, beginning with the computation of the elements c: 


C11 bin Bag bi4 Ist step 
C91 3 f50 bos bo, 2nd step 
C33 : C30 C33 bs4 3rd step 
C41 : C42 C43 


TasBLe V. Compact Scheme for the Single-Division Method 


= 

03 | 292 
0.5 2.68 
0.7 2.78 


0.22 0.9 3.22 

0.54 0.3 2.92 
0.82360|1| 0.11316 | 0.19767 | 0.45410 | cee 
(5 


0.09320 | 0.69785 — 0.22185" 0.71030* gs 
0.16280 9, —0.15482 | 0.49787] 1 1.48240 2.48239 


(40) 
1.48240 PA8239 


(7) 
1.03937 2.03916 
0.04348 1.04348 


(80) 
— 1.25780 | — 0.25779 


Horizontal sums in superior position. 


* Obtained by rounding to five decimals before performing division, which renders the procedure consistent with that af Table I. In practice this is nut 
ordinarily bothered with, and is not here in the check column. 
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Here any element is obtained as the difference between the cor- 
responding element a and the sum of paired products: of the elements 
¢ located in the given row (at the left) and the elements 6 located in 
the given column (above). Of course in computing the elements 4 
it is still necessary to divide by the corresponding element ¢. 

Thus 


€43 = 443 —64)b13 — Coby 


0.22 —0.66-0.54 —0.16280-0.11316 = —0.15482; 


_ 4347 641014 —Cygba4 


b3, = oa 
_ 0.22 -0.54-0.66 —0,09320-0.19767 _ _ 4 oo gs, 


0.69785 


We shall say a few words about the control to be applied when 
computing by the compact scheme. Just as before we form a 
column of check sums and perform the same operations upon it as 
upon the column of constant terms. 

In this connection each number of the transformed column must 
coincide with the sum of the elements of the corresponding rows of 
the matrix B, augmented by adjoining the transformed column of 
constant terms. The matrix B is of course the matrix of the coeffi- 
cients of the system obtained from the given one after completion of 
the forward course by the single-division scheme. 

In computing by the compact scheme it is convenient to use a 
triangle or two rulers in order to fix the attention on the required 
row and column. 


1 (Note that in Table V the units entered stepwise complete the tabulation of 
matrix B by filling in its diagonal; these of course figure in the row-total of the 
check. The return course, the third panel of Table V, is conducted by utuhzing 
the same numbers 4,,,;., as were utilized in Tables I to HY, in accordance with 
formulas (9), (5) and (3) of §6, i.c., analogous to formula (9b). The units 
arrayed northeast-southwest in the table serve, as there, the same purpose of 
locating the appropriate 4,;.;,, these being found successively in successively 
higher rows of panel two here, as distinct from Tables I, II and III, where they 
were found at the bottom row of successively higher panels. ] 
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§9. THE CONNECTION OF GAUSS’S METHOD 
WITH THE DECOMPOSITION OF A 
MATRIX INTO FACTORS 


It was shown in § 1 that a system of 2 linear equations in n 
unknowns can be written in matrix form 


(1) AX = F, 


A being the given matrix, nonsingular, ¥ and & being columns com- 
posed of the values of the unknowns and of the constant terms, 
respectively, and which we shall speak of as vectors. 

The Gauss method, performed with a fixed order of leading cle- 
ments, consists in replacing the given system by an equivalent 
triangular system by combining the equations linearly; this reduces 
to combining linearly the rows of “land FF. In the course of its 
application, in using the single-division scheme, we are obliged to 
add to the elements of some rows elements proportional to the 
elements of the preceding rows, i.e., to effect upon matrix «1 ele- 
mentary transformations of Type 6b’ of Paragraph 10 of § 1. 

The result of several transformations of this form, as was shown 
there, is equivalent to a premultiplication of the matrix A by some 
triangular matrix of the form 


V1 0 a Ne () 
(2) 721 Pee «es _f 


Val Var ose Yan 
As the result of these transformations we arrive at a system with 
a triangular matrix 


1 by ... by 


0 oee ) 
(3) B : es 


0 0 cae, 


Thus /4=B, which is to say 4=/°-! B, and consequently the 
matrix A is factorable into the product of two triangular matrices. 
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The compact scheme realizes this factorization. Indeed, J! =C, 
where the elements of the matrix C are defined by formulas (2) of 
§ 8, for from these formulas it follows that 


fa i Bate 
455 = Cyt p> C151; = p> Cubij 2 BY 
(from the formulas for ¢;;), and 
it i a 
> 2 C1b,j, 46:55; = > Cibizy E<S 


(from the formulas for 4, ;). 
These last formulas state that 


A = CB. 


Since the diagonal elements of the matrix B are equal to unity, 
such a factorization is uniquc. 

The compact notation for the schemes of the Gauss method that 
differ from the single-division scheme are likewise related to the 
factorization of a matrix into the product of two triangular ones, 
but with another choice of the diagonal elements. 

We remark that the compact schemes fix the order of the elimina- 
tion, and they are therefore applicable only in case all the deter- 
411 42 
42; Aag 
the theorem of Paragraph 11, § 1. 

We shall show that in case the matrix A is symmetric, 


minants 4a), ,--., [4] do not equal zero, as follows from 


(4) bi = tt 


ci; 


We have, indeed, A=CB, A’ =8'C’, and, since A= A’, 


Co Cc 

C4, O ... O Peete nt 

7 Ci Cy 

CB= BC’ = Bt 0 con 0 Oo! ot 
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Hence, on the strength of the uniqueness of the factorization of the 
matrix A into the product of the two triangular matrices, it follows 
that 


y 21 fal 
11 11 
c,9 
B= 0 1 = 
con 
0 0 l 


Thus the elements of 8 are simply the elements of the matrix C, 
divided. This notwithstanding, the compact scheme requires n? 
transcriptions even for the case of a symmetric A. 


§ 10. THE SQUARE-ROOT METHOD '! 


In this section we shall show that in case the matrix of the 
system is symmetric, finding the solution may be rendered still 
easier, since in such case the matrix can be resolved into the product 
of two triangular matrices of which onc is the transpose of the other. 

Thus let 


(1) A= S'S, 
where 
Sit Sig Sin 
0 ie ne, 8 
(2) $= se ee 
0 6—O0 5 


Let us determine the elements of the matrix S. In view of the rule 
for the multiplication of matrices, we have 


G35 = SyiS1jtSoiSopt oo + FSiSij ESS 
st) 9 2 , , 
yg = SyptSgpt ++ $45, et i 


1 ‘T, Banachiewicz, [1]. 
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Hence we obtain the formulas for the determination of the Siz3 


t 
$11 = Vai1; y= —, 


Jan Sa as; p> SyiS3j 
(3) 55; = a3; > Siis i> I; Sij ———— J > a 
i=l Si 


$20, t>y. 


Furthermore, the solution of the system reduces to the solution of 
two triangular systems. Indeed, the equation 


AX =F, 
is equivalent to the two equations 
SK=F, SX=K., 


The elements of the vector A are determined by recurrence formulas 


analogous to the formulas for s;;, to wit: 


(4) i=, &, See OS, 


The final solution is found by the formulas 


fn 
k.— > Six 
k, 5 oe ry had | . 
(5) moh a= ica 


ii 

In the scheme the usual control is to be employed, wherewith in 
forming the check elements we involve all the clements of the 
matrix. The control equation is an analog of that of the compact 
scheme: 


n 
k= Ysytk,. 
ka 
With the square-root method, one has to record only the approxi- 
2 
mately 5 elements of the matrix § and the 2n components of the 


vectors K and X. 
We exhibit in Table VI the solution of our system by the square- 
root method. 


TABLE VI. The Square-Root Method 


0.66 
0.44 
0.22 
1.00 
0.42 054: | 0.66 


0.90752 0.10270 0.17939 
0.83537* | —0.18533* 
0.70560 


— 1.25778 0.04348 1.03917 1.48238 


— 0.25779 1.04349* 2.03917 2.48238 


Horizontal sums in superior position. 


0.3 
0.5 
0.7 
0.9 
0.3 
0.41211 
0.59336 
1.04597 


® Obtained by rounding radicand to five figures before extracting root, or by rounding numerator before division. 


2.92 
2 


(2) 
1.60173 


1.24340 
1.75157 
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The computation of the elements s;; (as also the elements &; and 
k;) is carried out by rows in sequence. Any diagonal element is 
calculated as the square root of the difference between the cor- 
responding element @ and the sum of the squares of all previously 
computed elements s located in the same column. A nondiagonal 
element 5;; is obtained by subtracting from the element a,; the sum 
of the products, row by row, of corresponding elements s taken 
from the columns with numbers 1? and j. This difference, once 


obtained, is divided by the diagonal! element of the row. 
Thus 


434 — 519514 — SogSag 


534 


533 \ 
0.22 —0.54- 0.66 — 0.10270-0.17939 
So pggegg — 0.18533. 


The return course is carried through by the analogous formulas (5). 

With a little experience, the computations, by formulas (3), (4) 
and (5), will go easily. 

In case the elements of the matrix are such that radicands of the 
expression for s,; are negative, one will encounter no substantial 
difficulties while using the square-root method. Indeed, in this 
case, in the row for which s?,<0, only pure imaginary numbers 
figure; operations upon them are nowhit more complicated than 
upon real numbers. 

Let us clarify this statement with an example 


1.41421 — 0.70711 0.70711 2.82843 4.24265 


5B1i4i | —2.21360; | 4.427191 | —5.05965i 
2.32379 5.93858 8.26237 
111112 0.77779 | 2555956 | — 
2.11112 1.77779 3.55556 
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The square-root method is now widely employed where the solu- 
tion of symmetric systems is called for; it can be recommended to the 
reader as one of the most efficient methods. 


§11. THE INVERSION OF A MATRIX 


As has already been remarked in the introduction, the problem 
of solving a nonhomogencous linear system and that of inverting a 
matrix are closely connected with each other. 
Indeed, if for the matrix A its-inverse, A-!, is known, then on 
multiplying the equation 


(1) AX =F 
on the left by -1-!, we obtain 
(2) X = AM, 


Conversely, the determination of the clements of the inverse 
matrix may be reduced to the solution of 2 systems of the form 


pS Lesage 
(3) Saucy; = 6 
hol 


vi] 
t = l, > © @ > f, 


where 0;,; is Kronecker’s symbol. 
The last follows from the definition of the inverse matrix: 


AA-!1 =] 


and the rule for matrix multiplication. 

A technique employed in structural mechanics for determining 
the solution of a system by means of the so-called influence 
numbers! is none other than the solution of the system by construc- 
tion of the inverse matrix. The influence numbers themselves are 
the clements of the inverse matrix. 

The solution of the 2 systems necessary for the determination of 
the x? elements of the inverse matrix we shal] perform by the scheme 
for several equations, as shown in Table VII. As the result we will 
obtain a matrix consisting of the rows of the inverse matrix, arranged 


1A. A. Umansky [1]. 
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in counter-order. For control of the computation and an estimate 
of the accuracy of the result, the multiplication of A by A~! is 
expedient. : 

The theorem concerning the factorization of a matrix into the 
product of two triangular matrices makes possible the construction 
of a compact scheme for computing the elements of the inverse 
matrix.! The scheme requires 2n? entries in all, of which n® entries 
give the elements of the inverse matrix. 

Let 


(4) A = CB, 


where the elements of the triangular matrices C and B are deter- 
mined by formulas (2) § 8, which we reproduce here: 


jal 3 : 
6:3 = 4j;—- 2 Cybij, 12 J; 
fol 


i-1 
ai 2 6164; 
b,,; =—_——_,, i< 736, = 1. 


Cis 


Let us denote the elements of the inverse matrix A-!=D by d;;. 
We have, obviously, 


(5) D = B-1C-}, 


We shall show that the elements d;; can be determined without 
inverting the matrices B and C, 
Multiplying equation (5) by C on the right, we obtain 


(6) DC = B-1, 


The matrix B-! is obviously also triangular, with units along the 


n{n-+ 1) 


principal diagonal. We know, therefore, = of its elements: 


will be null and the remaining n will be units. 


Multiplying equation (5) by B on the left, we obtain, analogously, 


(7) BD =C-), 
1 F, Waugh and P. Dwyer [I]. 
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Since the matrix C-! is triangular, ae D of its elements will be 
zeros. 

It is readily seen that the system obtained by combining the 
ae) cquations of the system DC = 8-! that were mentioned above 


with the 


equations of the system BD=C-! is a recurrent 


nin—1) 
2 
system that makes possible the determination of the n? elements of 


the inverse matrix. 
We give it in extenso for n=4: 


61145) tarda t+egidigteaidig = 1 0 


Co CO Bw 


Cxod in + Cyodj3+Cgadig = I 


CoO OF OC & 


C394 ;3+C434;4 = l 


€43d;4 = 


4 
dy gt Bynds; + by gd3;+ bya; = 0 0 
dy, + beads, +ba4d4; = 0 0 

dy ,+bgqdy; = 0 


From the equations of the first group for 1=4, d44, d43, dyo, dq, are 
determined in succession. Next d3,, dos, d)4 are determined from 
the cquations of the second group for j=4. Following this the 
process goes on analogously and we use the formulas of the first and 
second groups in turn. Explicitly, dy), dg2 and d3g are determined 
from the equations of the first group for :=3, and d., and dj, are 
determined from the equations of the second group for j7=3; ds, 
and ds. are determined from the equations of the first group for 
1=2, and dj. from those of the second group for j=2; finally, d;, is 
determined from the equations of the first group with :=1. The 
inversion of the matrix by the compact scheme is exhibited in 
Table VIII. 


Tasre VII. 


0.42 0.54 
1.00 0.32 
0.32 1,00 
0.44 0.22 
0.42 0.54 
0.82360 0.09320 
0.09320 0.70840 
0.16280 —0.13640 
1 , 0.11316 
0.69785 
— 0.15462 
1 


0.66 
0.44 
0.22 
1.00 
0.66 
0.16280 
— 0.13640 
0.56440 
0.19767 
— 0.15482 
0.53222 
— 0.22185 


0.49787 


~ 0.57698 
— 0.70570 


— 0.68624 


— 1.37835 
— L.OLL49 
— 0.12304 

2.50759 


ojo co- 6 


1.21418 
—O.11316 
— 0.19767 
— 0.16216 


— 0.44747 
— 0.26143 

1.33221 
=, 12505 


Inversion of a Matrix: Single-Division Scheme 


0 
0 
} 
0 
0 
0 
J 
¢ 
0 
J 
0 
1. 


43297 


0.22185 


0.44560 
1.53183 
— 0.26142 
— 1.00149 


- Oo 9 8 


on So10 — 6 80 


z 
3.62 
3.18 
3.08 
3.32 
3.62 


1.65960 


1.12520 


0.93740 
0.60275 


(6 
1.39327 


(0) 
0.61072 
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0.09320 
0.16280 


Tasie VIII. 


0.54 
0.32 


1.00 
0.22 
0.54 
0.11316 
0.69785 
— 0.15482 


0.66 
0.44 


0.22 
1.00 


| 0.66 2.50759 


0.19767 — 0.12304 


1 | -0.22185¢ ~ 1.01148 
| 0.49787] 1; — 1.37834 


Compact Scheme for Inverting a Matrix 


1.33221 


i D 
—0.26143 


— 0.44745 


— 0.12304 


| 


| 


— 1.01149 


— 0.26142 


= 4" 53183 


0.44560 


xu vo fo uorssaauy 3y 7 
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§12. THE PROBLEM OF ELIMINATION 


This problem, in the simplest case, consists of the computation 
of the values of a linear form 


(1) €4%) HooXot ++: HEX ny 


where ¢;, ¢y,..., €, are given numbers and x), x2, ..., 4, is the solu- 
tion of the system 


Qy\X) + 4yo%0+ +++ +4y,x, = by 


(2) Q2\Xy+agoXe+ +++ +a xX, = de 


ee @  @ tf  @  @  @  @® @  @  @  @  @  @ 


Qn1*) + 4,o%9+ + °° +0,.%, = b,, > 


whose determinant is different from zero. 

The natural procedure for solving this problem consists in deter- 
mining the numbers x;, x9, ..., x, in explicit form and substituting 
them in expression (1). This may be avoided, however, as follows. 

Let us tabulate the matrix of the coefficients of system (2), 
adjoining the column of constant terms to the right of it, and under- 
neath it a row consisting of the coefficients of the linear form to be 
computed, the signs of the coefficients being reversed. We shall 
put 0 in the lower right corner. We shall have the scheme 


44; aye a1, by 

Go, age Qn by 
(3) =e @ @ @ @  @ @  @ 

| a,9 os ayn, b,, 

—C) —lo Fa ==, 0 


or, in abbreviated notation, 


(3’) ——————er 


The Problem of Elimination 9] 


Let 71, Yo, -- +» 7, designate the solution of the system of equa- 
tions 
QnYitaayat oo* $4, = 61 


o + daooVat +°* +4,0¥, = Ce 
(4) aoV1 T a4a9Ve2 annoy 


41,714 4a,¥2+ _ ae + BanYn = Cn 


Then 
biyitbayet +++ thay, = 


= (ayyxyt ss > +4 4y%,) 71 
+ (doyXy+ +++ +40,%,) Yo 
a ee 
+ (4,1%) + 11 + Ganka) Zn 

=  (4y~1 + ++ > +4,1%q) *1 


+ (ay07) a ee + 4,2Y,)Xe 
+ . 


+ (4a,,¥1+ ila + anPn)Xn 
= C1 X%1 Hoax + eee +¢,%,.- 


Thus the computation of the form ¢)x; +¢o%2+ +--+ +¢,x, may be 
replaced by the computation of the form 6,7, +doyo+ +--+ +,y,- 
On the other hand it is obvious that if we multiply the first row of 
scheme (3) by +;, the second by yo,..., the nth by y,, and add to 
the last row, we will obtain a row whose elements situated beyond 
the double line are equal to zero; the element in the lower right 
corner obviously equals the number that is sought: the value of the 
linear form. Conversely, if we somehow choose a linear combina- 
tion of n rows such that the addition of it to the last row gives a null 
row (up to the line), the coefficients of this combination will be 
solutions of system (4), and accordingly the element in the lower 
right corner will equal the sought number. This follows from the 
uniqueness of the solution of system (4). There is thus no need to 
find the numbers y), y2,-.., y,- Itis merely necessary to ‘‘annul 
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the last row” by the addition of a suitable linear combination of the 
first n rows. This can be done by the ordinary forward course of 
the Gauss process, applicd to scheme (3). 

Just such a device may obviously be employed for the computation 
of the non-homogencous linear form ¢,x; + «++ +¢,x,+4. There will 
be merely this difference, that in the lower right corner one must enter 
not 0 but the constant d, so that the initial scheme has the form 


411 F120 eee Gin | by 
(3”) 

G51 4,2 - Gan b,, 

—¢j — lo eee m0 | d. 


The solution of system (2) also may be found by this method, 
without employing a return course. Indeed, the cxpressions 
Xy) Xv, ...,%X, are particular cases of linear form (1), with coefh- 
cients (1,0,...,0), (0,!1,...,0),...,(0,0,..., I). Their 
determination may be effected simultaneously by the elimination 
scheme by simultancously entering in the lower left corner the rows 
(-—1,0,...,0), (0, —1,...,0),..., (0,0,..., —1), which col- 
lectively constitute the matrix —J. In the lower right corner, 
rather than the number 0, we place a null column. 

The source scheme for the solution of the system thus has the form 


A | 


(6) =. 
—l 


Having obtained a null matrix in the lower left corner of the scheme 
by the addition of suitable linear combinations of the first n rows, 
we will have in the lower right corner a column of the values of 
the unknowns. 

The inversion of a matrix is equivalent, as we have seen, to the 
solution of n systems of a special form, whose constant terms form the 
unit matrix. Their collective solution may be effected by means of 
the scheme 


A [ 
(6 =| ; 
_ I - 
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where J, as before, designates the unit matrix, and in the lower right 
corner is arrayed a null matrix of the nth order. After annulling 
all the rows in the lower left corner, by the addition of suitable 
linear combinations of the first n rows, we will obtain the matrix A~! 
in the lower right corner. 

The result of the elimination may be cast in matrix form. This 
makes it possible to obtain some generalizations. To wit: the solu- 
tion of the system, x,,..., x,, may be written in matrix form as the 
column 


= A-16, 


and the value of the linear form ¢,x,;-+¢oxe+ --++ +¢,x, as the 
number! cA-16. 

Such a representation points the way to the computation of the 
more complicated expression CA-!B, where C and B are certain 
rectangular matrices, of which C consists of x columns and B of n 
rows, and the number of rows of C and columns of B is a matter of 
indifference. Indeed, the element of the ith row and Ath column 
of the matrix CA~1B is the number ¢,1-1,, where ¢; is the ith row of 
the matrix C, and 64, is the kth column of the matrix B. 

The computation of the elements of the matrix CA-1B may there- 
fore be carried out by the method of elimination, applied to the 
scheme 


(7) es 


After annulling the elements located in the lower left corner, by 
adding a linear combination of the first n rows, we obtain in the 
lower right corner the matrix CA-!B. 


1 [It is to be remarked that the author's notation regards the row position as 
normal for a premultiplying vector, and the column position as normal for a post- 
multiplying one, no further diacritical marks thus being called for.] 
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Putting C= J, we obtain the scheme 
A 


(8) = 
-1 


for the computation of the product A-)B. 

By transposing the scheme, one can obtain a modification of the 
method of elimination, to wit: if D=CA-1B, then D’=B'(A’)-1C"; 
hence it follows that D’ (and accordingly, D as well) may be com- 
puted by a forward course applied to the scheme 

A’ 
(9) = 


In particular, for computing the value of the linear form 
€\X)+ +--+ +0¢,x, one may also use the scheme 


where 4’ is the matrix that is the transpose of that of the coefficients 
of the system, ¢’ is the column composed of the coefficients of the 
linear form being computed, 8’ is the row of constant terms, to be 
taken with signs reversed. The validity of this construction may 
be readily proved directly, without relying on the results set forth 
above, for if we multiply the first 2 rows of scheme (9) by x,,...,%, 
and add to the last, we will obtain zero in the lower left corner and 
€)X, +--+ +¢,x, in the lower right corner. 

Analogously, for solving the system A. = we may use the scheme 


I 


——_— 
—<—» 
e 


0 


al’ 


— §’ 


(10) 


We exhibit in Table IX the solution of the system, carried out by 
this modification of the method of elimination. 

As usual, the last column of Table [X is the control; throughout 
the duration of the process each number obtained in it must coincide 
with the sum of the preceding numbers of the same row. 


Taste IX. Solution of a System of Linear Equations by the Method of Elimination 


x; Xs * | 7) = 
~ 100 | 0.42 a ee ee 0 0 3.62 
0.42 1.00 0.52 0.44 1 0 3.18 
05+ =| 032 1.00 0.22 0 0 3,08 
0.06 0.44 0.22 3.00 0 1 -9,32- 
-0.3 -0.5 -0.7 -0.9 0 0 “| -2.40 
1 0.42 oss | 0.66 1 - oO 0 3.62 

0.82360 0.09320 0.16280 -0.42 a 0 1.65960 

0.09320 0.70840 - 0.13640 -0.54 0 0 1.12520 

0.16260 —0.13640 0.56440 — 0.66 0 1 0.93080 

— 0.37400 —0.53800 0.70200 0.30 o 0 = 1.31409, 

1 0.11316 0.19767 —0.50996 1.21418 0 2.01506 

0.69785 0.15482 = 0.49247 0.11316 “9 | 0.93740 

— 0.15482 0.53222 — 0.57698 —0,19767 l 0.60275 

— 0.49568 — 0.62807 0.10927 0.45410 0 - 0.56037, 

N — 0.22185 — 0.70870 -0.16216 1.43297 0 1.34327 

0.49787 0.22185 l o.ain2’ 

— 0.73804 0.71029 0 0.10546 

1 0.44560 2,00856 1.62838" 

1.03916 1.48240 1.30727 


1q fo w9jgorg IT, 


etUzt 


jot 


e 


Udi 


G6 


TasLe X. Computation of a Product A-'B 


1.00 
—0.85 
0.90 
0.60 
—0.80 
~y | of | 0O5¢ | 0.66 To dno 
0.16280 | —0.42 
0.13640 | —0.54 
0.56440 | —0.66 


— 0.68509 0.25 


a el 


1.65960 
1.12520 
0.93080 
— 1.29500 


—0.70200, 0.30 — 1.31400 
— 0.50100 0.15 — 0.95700 
— 0.66800 0.20 | | Bsr 
! | 0.11316 «6©| 0.19767. | —0.50996 1.21418 | 2.01506 
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0.69875 


— 0.15482 

— 0.47596 

— 0.49568 

— 0.34218 

— 0.45624 
l 


—— 


— 0.15482 


0.53222 
— 0.61680 
— 0.62807 
—0.45415 
— 0.60554 
— 0.22185 

0.49787 
— 0.72239 
— 0.73804 
— 0.53006 
— 0.70676 

l 


— 0.49247 
— 0.57698 
0.07406 
0.10927 
0.02914 
0.03885 
—0.70570 


— 0.68624 


— 0.26182 
— 0.24053 
— 0.21234 
— 0.28312 
~ 1.37835 


— 1.25753 
— 1.25781 
— 0.94295 
— 1.25728 


— 0.11316 
— 0.19767 
0.41889 
0.45410 
0.28776 
0.38368 
—0.16216 
— 0.22278 
0.34171 
0.37372 
0.23227 
0.30970 
— 0.44747 


0.01846 
0.04347 
— 0.00492 
— 0.00655 


(A-'By 


] 
0 
0 
0 
0 
0 


1.43297 


0.22185 


0.68204 
0.71029 
0.49033 
0.65378 
0.44560 


1.00394 
1.03916 
0.72652 
0.96871 


(5) 
1.41957 | 1.12448 


0.93740 
0.60275 


— 0.47943 
(5) 
— 0.63924 
1.34327 
0.81072 
0.03954 


1.45096 
1.48240 
1.06466 


uorpururpsy fo wajgorg 7Y LT 


L6 
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Observation. In solving systems by the method of elimination 
the number of operations somewhat exceeds that for the Gauss 
method. The uniformity of the process, however, and especially 
the absence of a return course, not infrequently make this method 
the more convenient one. 

For inverting a matrix the scheme 


| 


A’ 


yf 
(11) 
~1 || o 


may be used. Here, after completion of the process, we will obtain 
in the lower right corner a matrix that is the transpose of 4-1. 

Thus for each of the problems analyscd we have two schemes, 
defined cither by the matrix A or by its transpose. We note that it 
is most efficient to use that scheme which contains the least number 
of rows in the lower left corner. Thus in the problem of solving a 
linear system without a return course it is most efficient to use the 
scheme with the transposed matrix; in the problem of computing 
the product 4-!8B—the scheme with the transposed matrix in case 
the number of columns in B is less than n; the given matrix should 
be used as such if the number of columns in 8 is greater than n. 

In Table X we exhibit the computation of the product 4-!B. 
Here, as A, we have taken the matrix 


1.00 0.42 0.54 0.66 
0.42 1.00 0.32 0.44 
0.54 0.32 1.00 0.22 
0.66 0.44 0.22 1.00 


and as B, the matrix 


0.25 0.30 0.15 0.20 
0.45 0.50 0.30 0.40 
0.65 0.70 0.45 0.60 
0.85 0.90 0.60 0.80 
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The computation of A-'B has been effected by scheme (9) with 
C’=I, We have as the result of the computations 


— 1.25753 —1.25781 —0.94295 —1.25728 
0.01846 0.04347 —0.00492 —0.00655 
1.00394 1.03916 0.72652 0.96871 
1.45096 1.48240 1.06466 1.41957 


A-1B = 


§ 13. CORRECTION OF THE ELEMENTS OF 
THE INVERSE MATRIX 


Inversion of a matrix by any of the schemes exhibited above 
offers no certainty as regards the accuracy of the results obtained, 
owing to the inevitable rounding errors, the influence of which on the 
final result is hard to gauge. For a check of the accuracy of the 
matrix Dy obtained from the given matrix A by whichever process 
of inversion, one should determine the product ADg; the discrepancy 
between it and the unit matrix will indicate the degree of inaccuracy 
in the results obtained. 

Let this check computation reveal that the approximation Dy to 
A-) is such that j|Foli<k <1, where 


(1) Fy = [-ADo. 


The first or second norm of those introduced in § 5 is conveniently 
taken as the norm of this matrix, they being the most readily 
computed. 

Under this condition the elements of the inverse matrix, A-!, may 
be computed to as high an accuracy as is desired by means of the 
following iterative process.! 


1H. Hotelling, (1). We note that Hotelling took as norm the expression 


{ 
; > (ay)? » for which the requirements of § 5 are not satisfied. In § 17 the 
~ife1 


relation between this expression and the third norm is explained. 
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Let us form the sequence of matrices 
D, = Do(f+Fo), F, = I-AD, 


2 = D,(I+F;), Fe = 1-ADy 
(2) 


D,, = Dy-\(I+F,-:), F, = 1- AD, 


We shall show that the matrix F,,=/—AD,, is equal to F?”. 
Indeed, 
F,, = 1-AD,, = 1-AD,_\(1+F y-3) 


(3) = 1-(1-F,,-1) (2 +Fy-1) 

= FU, = Fle =-:+ = Fy. 
Hence it follows that 
(4) D,, = A W(I-F"). 


The last formula shows that D,, approaches A-!, the convergence of 
the process being very rapid. Let us make an estimate of the 
error, taking into consideration the fact that A-!=D)(ADo)-! 
= Do(f-F)-!: 


||D,— A“ = [| - AF "|| = ||- Do(— 8) “Fo 
on 


om k 
(5) < ||Doll 7-H) Fol < Doll p=" 


It is seen from this estimate that, if only the iritial approximation 
be so chosen that ||/— ADo||<’ <1, the number of correct figures of 
decimals increases in geometrical progression. 

The successive approximations should be computed by lifting the 
parentheses in formulas (2), viz.: 


(6) Da =e D+F, -1) Pa Dae + D,,.\(1— AD,,_;). 


The second summand will here play the role of a small correction 
to the first. 

Sometimes, starting with the matrix Fo, it is expedient to form the 
matrices F, =Fj, F, =F§=(F,)*, squaring the matrix Fisuccessively, 
afterwards utilizing formula (6). 
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Observation. In case A is a symmetric matrix, and one has taken 
as the initial approximation Do a symmetric matrix, all the subse- 
quent approximations will likewise be symmetric matrices, although 
the matrix Fy may turn out to be non-symmetric. Indeed, from 
formula (6) it follows that D,=2D,_,—D,-;AD,,_1, whence, on 
assuming that A’=4, D’_,=D,,_1, we obtain 

Di, => 2Din-1 —(D,-);AD,-1)' = 2Din-1 — Diy A'Dn.; 
= 2D,,-1-Da-ADp-1 = D,,- 
As an example, let us invert the matrix 
1.00 0.42 0.54 0.66 
0.42 1.00 0.32 0.44 
0.54 0.32 1.00 0.22 
0.66 0.44 0.22 1.00 


As the initial approximation let us take from Table VIT, § 11, the 
result of the inversion of the matrix A by the single-division method 
(retaining four figures after the decimal) : 


/ 2.5076 —0.1230 -1.0115 —1.3783 
— 0.1230 1.3322 —0.2614 —0.4475 
—1.0115 -—0.2614 1.5318 0.4456 
— 1.3783 —0.4475 0.4456 2.0086 


Dy, = 


The check computation gives, for Fg, the following value: 
—52 —18 20 —50 
— 60 8 —-—10 10 
—-18 —3+ 26 -10 
— 66 20 10 —54 
Hence we see that '!Foi!;<¢0.000150, iF y!|,,< 0.000196. For the 


estimate of the error Ict us take ‘lFo/,. On the strength of formula 
(5) we have, taking into account that || Do||, <5: 
- (0.00015)- 


— 4-H eu te ie 
D1— AN <9 P—=90015 


Fy = 10-6. 


< 0.000000). 
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Thus D, gives for A-! a value correct at least to a unit of the 
seventh decimal place for each of its elements. 
Doing the computation, we obtain 
,2.50758616 —0.12303930 —1.01148870 — 1.37834207 
—0.12303930  1.33221281 —0.26142705 —0.44745375 
— 1.01148870 —0.26142705 1.53182667 0.44560858 
— 1.37834207 —0.44745375 0.44560858  2.00855152 


The check computation /— AD, gives 


20 0 1} 

1o-l } 
I—AD, = 10-8. 

10 oO 0 

1o oO -1 


§14. THE INVERSION OF A MATRIX BY 
PARTITIONING 
It is sometimes expedient to partition a matrix beforeinversion. 


Let us examine the formulas for inverting a matrix of the nth order 
partitioned into four cells by the scheme 


where A and D are square matrices of orders p and g; p+q=n. 
Let us seck the inverse matrix also in the form of a cellular 
matrix: 


K and N being again square matrices of orders p and q. 
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In conformity with the rule for multiplying partitioned matrices, 
the following matrix equations must hold: 


AK+BM = 1 


AL+BN =0 
CK+ DM = 0 
CL+DN = 1. 


Multiplying the third equation on the Icft by BD-1 and sub- 

tracting it from the first, we obtain 

(A-BD-'C)K = |, 
whence 

K = (A-BD-'C)-1, 
Furthermore, from the third equation we find that 

M = —D-'CK. 

In like manner we find from the second and fourth equations that 


N = (D-CA-1B)-! 
and 
Lh= -A-'BN, 


Of course these formulas have been derived on the assumption that 
all the indicated matrix inversions are realizable. 

The inversion of a matrix of order 2 thus reduces to the inversion 
of four matrices, of which two are of order p and two are of order 4, 
and to several matrix multiplications. 

The formulas developed above can be altered so that for the 
computation of the matrices A, L, Af and N only two matrices, of 
orders # and q, nced be inverted. For, as can readily be verified, 


N = (D-CA-'B)-!,) ML = —NCA-), 

L= -A'BN, K = A-'-A- BM, 
and, analogously, 

K = (A-—BD-'C)-!, £ = —KBD-!|, 

M = —D-'CK, N= D-'!-D-\CL. 
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The last formulas show that the method of partitioning is con- 
veniently employed when any diagonal cell is easily inverted. 
Let us take as an example the inversion of the matrix 


100 0.42 : 0.54 0.66 
0.42 1.00 : 0.32 0.44 


COO e Ceo eo err Ce eOOe COE EED ESOS CELE EEE 


0.54 0.32 : 1.00 0.22 
0.66 0.44 : 0.22 1.00 


The computation will be performed as follows: 
1) We compute the matrix 4-!: 


jn 1.21418 —0.50996 
~ \ —0,50996 _—«'1.21418 
and form the products 
0.49247 0.57698 re ew 0.11316 
0.11316 0.19767)’ ~ 1 0.57698 recor | 


0.30215 0.37482 
0.37482 0.46778 } 


AB = ( 


CAB = ( 


By twice computing the last matrix, as C(A-!B) and (CA-!)B, we 
obtain a check on the preceding computations. 
2) We form the matrix 


(D—CA-1B) = ( 0.69785 popvinel 


— 0.15482 0.53222 
and find its inverse 


1.53183 0.44560 
= —-CA-!B)-1 = ; 
N = (D-CA-!B) ( 0.44560 2.00855 


3) We compute the matrices 


—1.01148 —0.26142 
M = —NCA-) = ( — 1.37834 —0.44745 


—1.01148 —1.37834 
= —Jf-1 = 
ae Ean eg 


The Bordering Method 105 
and 


2.50757 —0.12305 
K = A-!—A-1BM = ( 


— 0.12305 1.33221 
The sought inverse matrix will thus be: 
2.50757 —0.12305 —-—1.01148 —1.37834 
— 0.12305 1.33221 —0.26142 —0.44745 


—1.01148 —0.26142 1.53183 0.44560 
— 1.37834 —0.44745 0.44560 2.00855 


S-l = 


§15. THE BORDERING METHOD 


In this section we shall consider computational schemes for the 
inversion of a matrix and the solution of a linear system that are 
based on the idea of bordering. 

The given matrix A will be regarded as the result of bordering a 
matrix of the (#—1)th order, the inverse matrix of which will be 
considered to be known. Thus let 


Qi] a\o ay, n-1 ai, 

ae G22 a2, a-1 a2, 
(1) A, = 

Gn-1,1 %,-1,2 an—-1, n-1 Gn-1,n 

Qn} a,2 an, n-l aon 


Here A,_,; denotes the aforementioned matrix of the (n—1)th 
order, 


G15 
(2) u., = (4,1; ceey Gy. a-1)> “un, = ( : : 


Gn-1. in 
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We seek the matrix A-! as a bordered matrix too, in the form 


P,-1 ln 
(3) p= Ass =| i) 
Gn or 


: : ! 
where P,_; is a matrix, q, a row, r, a column and — 2 number, all 


of which we have to determine. 
By the rule for multiplication of bordered matrices we have 


A,-1 u, Feat r, 
AAW} = 
v, Gon Fp, a, 
Uy 
A,-iP -] + 4,491) Anat e ( ne 
UP a1 + 4,49 q Ugh y tet o 
Hence 
(4) < Pee ee +U,9, =1 
(5) U,P 1 + 4nnGn = 0 
u, 
(6) Ayn te =0 
Gan 
(7) (tS = I. 


We have from equation (6) 


~1 
Anata, 
a 


(8) = 


Substituting this value of r, in (7), we obtain 
(9) q, = Big — Up Ag Uy: 
Furthermore, we have from (4) 


(10) Piiy = Apt — Ag adn 
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and therefore, on the basis of (5) and (10): 
Dy Ag — Uy Ag Un Fn + Fann 
= nA, — (Gan %n) Int Fann = Un +2 9n = 9. 


Hence 
(11) poe 
a 
Finally, 
(12), ae = AT 1+ ie tata 1, 
Thus, at last, 
Az}, 4 4a=1 Mend 1 — Ags 
a, a, 
(13) Avis : 
uA ] 
, ot, 


where a, =a,,—U, 4,1, u,. 

The constructed formula is obviously a particular case of the 
formulas for inverting a matrix by the partitioning method, # here 
equalling n—1 and g being |. 

The method developed here is basically a method of inverting a 
matrix by successive borderings: one constructs in succession the 
matrices inverse to the matrices 


Qi, @2 443 


Qy,  4y2° 
(441), ’ Gv, Ann Ayzy fy--- 
Gy, Gee | 
@31 Ayn 433 

of which each successive one is obtained from the preceding by 
bordering. Each step of this process is accomplished on the basis of 
formula (13), viz.: if A;7_', is already known, the following operations 
must be carried out in order to find Ay!: 

1) The computation of the column: —4,_',u,. The elements of 
the column, f,,...8,-1, » are found by accumulation. 

2) The computation of the row: —v,Av7!, with clements 


Yat>*>> Ya, a-le 
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3) The computation of the number 
n--] a | 
Cc, = Ang + 2 FniBin = Qnn t 2 BinY nie 
te i=] 


(The computation twice of the number a, is a good check on the 
preceding computations.) 

4) Lastly, the determination of the elements d;, of the inverse 
matrix by the formulas 


di, = a, + Yaibe (i, k<n—1) 
(14) 
d,, = bin, dy = T# (i, k<a—l), d,, = 


] 

Rg n = a, 
Here dj, arc the clements of the matrix 47). 

In case of a symmetric matrix the scheme is obviously abridged 
by exactly onc half. 

In Table XI the inversion of a matrix by the mcthod of successive 
bordering is exhibited. 

Each step of the process is set up on the following scheme: 


—1 -1 
A 1 | uy, . A, yen 
15 
( Vy, ann 
-1 3 
—vAT' : a, 


The method of bordering may also be useful in application to the 
solution of a system of linear equations. It is especially convenient 
to utilize this method in case it is necessary to solve a system for 
which the truncated system—obtained by striking out one equation 
and one unknown—has been solved earlier. Such a situation is 
often encountered in applications. For example, in solving prob- 
lems of mathematical physics by the method of B. G. Galerkin or 
Ritz, it may occur that the solution obtained as the result of utilizing 
the (n—1)th coordinate function is insufficiently exact; if the 
addition of just one more coordinate function is sufficient for the 
construction of a more exact solution, the new system for the dcter- 
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TABLE XI. dnverston of a Matrix by Bordering 
Scheme: 
1.00 0.42 0.54 0.66 Amt | uy | — Az! n= [Bial 
0.42 1.00 0.32 0.44 Cn @nn 
| 


—tadg?y=(Yn) an =4na—tadg- itn 


—$—ar 


ee 


1.00 0.42 — (),.42 Formulas: 
n-—1 -1 
0.42 1.00 On = any, + Pa Gah in=4ant f: Binz'nt 
- -1 
— 0.42 0.82360 Av?) =Da-1=(di) 
ee Fis = D,= [dy] 
1.21418 —0.50996 | 0.54 — 0.49247 nil! 
) dy=dy te (i k<n—-1); 
— 0.50996 1.21418 | 0.32 —0.11316 P an 
Pin, =ink (; hen—l): 
0.54 0.32 1.00 Ama oaks RSE); 
1 
— 0.49247 —0.11316 0.69786 dan = — 
Be Ne es Pe a Pe eee ee n 
1.56171 —0.43010 —0.70569 | 0.66 — 0.68623 
— 0.43010 1.23253 —0.16215 0.44 — 0.22277 
—0.70569 —0.16215 1.43295 0.22 0.22185 
0.66 0.44 0.22 1.00 
— 0.68623 —0.22277 0.22185 0.49788 


a EE eek Se Re 


2.50754 —0.12306 —1.01147 —1.37830 
— 0.12306 1.33221 —O0.26141 —0.447-4+4 
— 1.01147 ~o261414 7 1.53180 0.44559 
— 1.37830 —0.44744 0.44559 2.00852 
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mination of the coefficients is obtained by bordering the previous 


system. 
The method of bordering is applied to the solution of a system of 


linear equations in the following manner. 
Let the system have the form 


(16) AX, = F,. 


Let us employ the symbols 


mar(S th ne("h (2) 


Then 


‘es Ayu? An) —Azu, ( Foi 
a, —v,AT, ] Je 
( A, yF,~4 I] ( Ag UU Aga 1 — An 1%atn 


0 a, am vAT Fi -1 +f, 


| 


But 4,_'|F,-, is the solution of the truncated system, i.c., the system 
QyyXytice es $Ay p-1%p-1 =fi 
a,-1, tS i +4,_1, n—1*n-1 = fi-1 


which solution we shall denote by X,_}- 
Analogously A71,u, is the solution of that same system with the 
constant terms changed, that is, the solution of the system 


QyyXyt 05> $44, pa 1%_-1 +4, , = O 


Gn. Xa 88 4g), g—-1% 2-1 F 4g-1, a = 0, 


which solution we shall call Q,_}. 
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Knowing X,-, and Q,_1, we easily compute X,, for 


A. = (o + l ( a ae) 


X,,- Qn-1 
a Si v aA ns 1 m 
7 ( 0 "| See Qn-1 ( ] ) 


Thus for the determination of X, we must, in addition to X,_, still 
compute Q,_. 

If the truncated system had been solved by Gauss’s method, it 
would be necessary to have added yet another column in order to 
have found Q,_,. The computation of the forward course is 
effected only for this column, utilizing the left half of the scheme, 
already known. Afterwards Q,-, is determined by the usual 
return course. 


(18) 


§ 16. THE ESCALATOR METHOD 


In inverting a matrix by the bordering method, the essential 
role in passing from the inversion of the matrix A,_ to the inversion 
of the bordered matrix A, is played by the computation of the expres- 
sions — Ay 'u, and —v,A;_1). 

It is obvious that the components of the vector — Aj_',u, are none 
other than the solution of the system of equations: 


(0) Ay Zp FH AyoZogt °° + +4, gy gt ay = O 


Oya, ZF 44-1, 2204 8 Hg, 4174-1, 2-1, 4 = 9,7 


1 (-Ar!u, of §15 is a column vector with (4—1) components, say z,, where 
2h = (on ++ Sk-4, x} and thus 


— Agta = =, 
Uy = Ay Xy 
i.e., Ay yop tu, =. (101) 
But u, = {ay eee ay -1, x}. 


(101) is therefore the author’s (0)=(2) in matrix notation. Note that when 
k=n+1, (101) becomes the author's (1), with 2, 7. ,=%,-] 
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Analogously, the components of —v,4,_!, form the solution of the 
transposed system. 

The successive solution of such systems, for k=2,...,2, 2+], 
lies at the basis of the so-called escalator method for the solution of 
of systems of linear equations.!_ By the same token, the escalator 
method is closely connected with the method of bordering. 

The author of the method limits himself to imparting a recipe for 
constructing the solution, and to formulas for the case n=4. 

The author considers the essential merit of the method to lie in 
the existence of a reliable check, by means of which the accuracy of 
the computations can be regulated; in particular, the method gives 
a result that is sufficiently reliable even in case the determinant of 
the coefficients of the system is small. 

We shall set forth the escalator method only in application to 
systems with symmetric matrices, giving the computational scheme 
the compact form. The connection between the escalator method 
and Gauss’s method will thereupon be revealed. 

So let there be given a system of equations 


44)%,FOyoXet 62 $4 ky, a41 = O 
(1) $e Git desis abo epn sade Gn Me whew a n'a a 


GX) +4,oXo 08! HAyX aH 4n, 41 = O, 


the matrix of the coefficients of which is symmetric. 
Designate by 24,,..., 24-1, , the solution of the system: 


AyyZygb ies $4), 4-1 %y-1, tay = 9 
(2) . 


Opg, Zag te ot Hg, p12 24-1, e+ %-1, 4 = O. 


The numbers Zz; ,,,;=%; obviously form the solution of system 
(1). If, therefore, we establish a means of computing in turn the 
numbers z;, for :<k<n+ 1, we shall by that fact have given a method 
for constructing the sought solution. 

Let us first assume that all the numbers z;, have already been 
computed for 1<k<n. 


1 J. Morris (!}. 
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By means of them let us construct the matrix 


1 22 Zig ++ Zag 
0 ] 223+ + + Lan 
(3) 220 oO 1 cn & 


0 0 oO... | 


It is easily seen that the matrix AZ has zeros above the principal 
diagonal. Indeed, 


B13 814239 4+ Aya, «+ sy Ap Z1_ FAyoZo, + °° > +44, 


G21, 421212 +429, - + - 5 Ga121n + 4agZen F °° HG, 


ani 441212 + anos eeeg 441 Z1n + AnggZuq + oe +4,,, 


and, in view of the definition of the numbers z;;, all elements lying 
above the principal diagonal are equal to zero. The non-zero 
elements of the matrix C’, are computed by the formulas 


(5) Cig = GyiZ1gHGjaZag+ ++ $4;, 5-121, ;+4ijz. TZ). 


Hence follows the connection between this method and that of 
Gauss, when the latter is considered in the light of a resolution of the 
matrix into factors. 

Indeed, from (4) we have 


(6) A=(C,Z-1. 


But the matrix Z-! is a triangular matrix with unit diagonal and 
zero elements below the diagonal; C, is a triangular matrix with 
zero elements above the diagonal. Comparing this resolution with 
the factorization of the matrix corresponding to the single-division 
scheme of the Gauss method (A =CB), we obtain, on the strength of 
the uniqueness of such a resolution, that 


(7) Zl = B, C; = C. 
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Let us represent the matrix C in the form 
| 0 0 Ae “O C11 


Y21 ] 0 ... O Co9 
(8) C= Y¥31 =Y32 l ... O C33 = JA. 
Yall Yn2 Vn3 °+-> l Can 


Here 


Cj; 
za S, E 
Vij om t>J 
We shall show the matrix ['=CA-! to be the matrix that is the 
transpose of the matrix Z-!. In doing so we shall make essential 


use of the symmetry of the matrix A. We have : 
A=IFAZ1 =A = (2-1)'AT". 


But (Z~-!)' is a triangular matrix with zeros above the diagonal; J” 
is a triangular matrix with zeros below the diagonal and units on it. 
On the strength of the uniqueness of the factorization, J” = Z-), 

This circumstance permits us to give recurrence formulas for the 
successive determination of the numbers z;;. Let us assume, indeed, 
that we have already computed the clements of the first 4 columns 
of the matrix Z. Then by formulas (5) the first £ columns of the 
matrix C can be computed, and consequently the first 4 columns of 
the matrix J’, that is, the first k rows of the matrix Z-!. In order 
to continue the process we must compute the elements of the 
(A+1)th column of the matrix Z. Since the diagonal clement of 
this column equals unity, and the elements below the diagonal 
equal zero, it remains for us to give formulas for the computation 
of z; x41, where i<k. From the equation Zf’=J we obtain, on 
the strength of the rule for the multiplication of matrices, the follow- 
ing recurrence formulas: 


Vert, 1+ Ye41, aZj2 + **° + Yn41, 221g + 71,441 = 0 
(9) Mesa, 2b oo HY ass, 222¢+ 22, g41 = 0 


Vi+t, A+, 241 = Y, 
defining the elements of the (4+ 1)th column of the matrix Z. 
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The reliable check alluded to earlicr consists in the actual reduc- 
tion to zero of the subdiagonal] elements of the matrix C’.! 

We subjoin the compact scheme of the escalator method for a 
system of four equations. (Recall that for uniformity of computa- 
tion the constant terms are tabulated in the left parts of the 
equations.) 


bed 
~~” 
- 
Co 
= 
> 


1 [Since Zz41=(21, ng. - +> Sng.eneis=i%1- > + X,} =~ is the sought vector, the 
computation is conducted with augmented matrices A and Z as follows. Denote 
system (2) by Ax+4=0 for brevity, and form the augmented matrices 


~~ EN 


C=AZ= 


|: > oo |: : | & : ml C : r 
-h : ojto : 1 ~-h'Z : —h'x [ome ma 


Here r is the vector of residuals, which must equal zero to the limit of computa- 
tional accuracy; the trailing cell of C, —A’x, is not computed, being of no utility. 


“~. e . » e 
Of Cit remains true as of C that all elements above the principal diagonal are zero, 
and thercfore the reasoning that led to the equations for the determination of ¢,;, 
4, and z,,; remains valid. The following computor’s formulas summarize the results: 


Denoting by a, the ith row of 4, 
ay the jth column of A, 


we have, since C=AZ 


Cp = % 2G 
(10) So ¢j=4(; 2 for A is symmetric. 
(11) yy=t, i>j, 


Ci 
(12) z4 is yielded by <1 ¥=0 for i#j.] 


TABLE XII, 
1.00 0.42 0.54 0.66 -0.3 
0.42 1.00 0.32 0.44 —0.5 
0.54 0.32 — 1,00 0.22 —0.7 
0.66 0.44 0.22 1.00 —~0.9 
—0,3 —0.5 —0.7 —0.9 
1.00 0.42 0.54 0.66 —0.3 
1.00 0.11316 0.19767 — 0.45410 
0 0.82360 ' 
1.00 —0.22185  —0.71028 
0.000003 0.000003 0.69786 
‘ 1.00 — 1.48237 
0.000006 0.000005 —0,000001 0.49788 
1.00 
0.000010 0.000008 —0.000009 —0.000001 


o os oe 8 


1) 


2) 


3) 


The Escalator Method: Compact Scheme 


0.42  —0.49247 
1 — 0.11316 
0 1 
'Z 
0 0 
0 0 


Computing Formulas 


— 0.68623 
— 0.22277 
0.22185 
l 
0 


— 1.25776 
0.04350 
1.03914 
1.48237 
] 


ey=aty 2G where a; is the ith column of 4. 


ey 
ey 


= is yielded by 2;, y(i=0 for i#jwhere z is theith 


row of Z 


OTT 


suoyonbsy avaury fo suas 
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The scheme consists of three parts. In the first part is centered 
the matrix of the coefficients of the system (symmetric, under the 
condition). In the second part we gradually tabulate the elements 
of the columns of the matrix Z. In the lower part are entered the 
diagonal and subdiagonal elements of the matrix C’ and the ele- 
ments of the matrix J”. We note that the elements of the 4th row 
of the matrix I” are obtained by multiplying the columns of the 
matrix A by the éth column of the matrix Z and dividing the sums 
obtained by the clement ¢c,,. A column of matrix Z is filled up by 
formulas (9). In Table XII an illustrative example is given. 


§17. THE METHOD OF ITERATION 


We shall pass now to a description of iterative methods of 
solving systems of linear equations. These methods give the solu- 
tion of the system in the form of the limit of a sequence of certain 
vectors, the construction of which is effected by a uniform process 
called the process of iteration. 

Let the system of linear equations be given in the following form: 


X= AyyXy + Ayoxet °° + +4y,%, +h) 


(1) Xq = Gq1Xy + Aoor%et +++ +42.%, tf 
Ky = Gy)%) F4yoXot + °° t+ aan tty 


At first glance such a form of notation for the system may seem some- 
what artificial, if only owing to the presence of like terms. This 
form is, however, especially convenient for the application of 
iterative methods. In the following section a number of devices 
will be considered which transform the given system into form (1). 
Every such device will give some modification of the iterative 
method. 
Let us write system (1) in the form 


(2) X = AX+F, 


where A is the matrix of the coefficients, F the vector of constant 
members. 
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Let us construct the indicated sequence of vectors as follows: in 
the capacity of an initial approximation let us take a certain vector 
X(9, chosen, generally speaking, quite arbitrarily. Next let us 
construct the vectors 


X® = AXY47 


X2) = AXW46 
(3) 
XW = AXG-D4F 


If the sequence X 0), X(),,..,.X%,... has the limit X, this 
limit will be the solution of system (2), for on passing to the limit in 
the equation X®=AX-D+F as k->-o, we obtain X=AX+F, 
which proves our statement. 

We shall now give necessary and sufficient conditions of the con- 
vergence of the process of iteration, and also a number of sufficient 
conditions and estimates of the rate of convergence, based on the 
results of § 5. 


THEOREM 1. For convergence of the process of tteration with any 
initial vector X © and with any value of the vector F, it is necessary and 
sufficient that all proper numbers of the matrix A be less than unity in modulus. 

Proof. One can readily convince oneself that 


(4) XW) = ALXO)4([4A4 --- + AMI)F, 


Indeed, for £=1 this is true, and for the remaining & it is verifiable 
by the method of complete induction. It hence follows that for the 
convergence of the iterative process with any F it is necessary that 
the serics /+A+ +--+ +A‘-1+4+ -+- converge. Indeed let us take as 
X‘®) a null vector, and as F a vector all of whose components except 
the ith are equal to zero, the ith component being equal to unity. 
Then X® equals the :th column of the matrix 7+ A+ --- +Aé-!, 
For convergence of the sequence X‘) it is necessary and sufficicnt 
that all elements of the ith column of the matrix /+A+ -.- +A‘-! 
have a limit, and since i may be taken as anything from | to 2, the 
series [+A+ --+ +At-1+4 --- must converge. But this condition 
is also sufficient, for with it satisfied, A‘—0, and consequently the 
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first summand, A‘X(, in equation (4) tends to zero, and the second 
summand, ([+A+--- +A*-1)F has a limit equal to (J+ A+ --- 
+ Ak-14....)F. In §5 it has been shown that the necessary and 
sufficient condition for convergence of the series /+A+--- 

+At-14 ... is the inequality |A;| <1, for all proper numbers 4; of 
the matrix A. This proves Theorem 1}. 

Since the condition of Theorem | is difficult to verify, it is better 
to judge the convergence of the iterative process by means of some 
sufficient signs—those connected directly with the elements of the 
matrix. These criteria follow at once from Theorem 2, $5. 


THEOREM 2. In order that the process of tleration converge, it is 
sufficient that any norm of the matrix A be less than unily. 

Proof. Indeed, if ||A||<1, all proper numbers of the matrix are 
less than unity and therefore, on the basis of Theorem |, the iterative 
process converges. 

We shall now give estimates of the rate of convergence of the 
iterative process in terms of the norm. Here the selection of the 
norm of vectors is a matter of complete indifference, but the matrix 
norm must be subordinate to the vector norm one has chosen. 


THEOREM 3. /f||Al| <1, then 
(5) |X- XH] < [XO] ae4 FETE 
Proof. We have 
|X- Xe 
= ||I—A)IF— (1+ A+ - ++ ANF AbX OD 
< IU A)-1— UF AG ++ + AEN) + AEX | 
< |(E—A)-1— (LAH += + AEN) FIL + AIH X OL 


FU LLAllé 
< |[A lA || +-—— 
* 1 —|{All 

It is often important to compare the accuracy of two successive 
approximations, te., to compare the quantities ||¥—.X || and 
|X — X41], 
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Such a comparison may be made on the basis of the following 
theorem. 


THEOREM 4. [| X—-—X “|| <||Al] ||X¥—-X4-»]]. 
Proof. Indeed, from the equations 


X = AX+F 
X®) = AXG-D4F 


it follows that 
X-X® = A(X-Xt-0), 
Hence 
|X— XH] = A(X — XE] < || A |] [X—X4-V]]. 


The vector norms I, IT, III, introduced by us in § 5, and the matrix 
norms subordinate to them, give the following easily tested sufficient 
tokens of convergence of the iterative process and estimates of the 
rate of its convergence. 
Rn 
I. If > |a,;|<4<1, the process of iteration converges, and 
fol 
(6) [x;—4)| < pmax |x;—2'-), 
3 


where X=(x,,...,%,) and X® = (x), 2... xf), 


Il. if > |a; |<» <1, the process of iteration converges, and 
i=) 
(7) > |x; - +f? |< v> |x; —aff- |. 


IIT. If > az,< P<], the process of iteration converges, and 
isl 
kal 


@ PS wane VPLS (gate 


Indeed, the largest proper number A, of the matrix A’A does not 
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exceed the sum of all the proper numbers of the matrix A’A, for they 


are all non-negative. 
But 


Ate e+ +4, = tA’ = > (ay)? = P 
kal 


Therefore |!Alli4, =WA, < VP <1, and consequently 


|X — XO = 2 > (x; —x{)? <vVP > (x; —xff—)?. 


We shall indicate still another way of constructing sufficient 
tokens of the convergence of the iterative process. 
In the equation 


(9) X = AX+F 


with matrix 


(10) Wet tee & oad, 


an1 a1 eee a 


let us introduce new unknowns x; =/;z;, where p; are some positive 
numbers. 
System (1) will then be transformed into the system 


(11) bz; = 2 a; ;P;2; +S; 
or 
(12) => ay Gg z; +5 ei 


It is obvious that the components of the successive approximations 
X (4) for system (9) and Z‘ for system (12) will also be connected by 
the relations x) =p,z), provided these relations hold for the initial 
approximations X‘) and Z‘). The iterative processes for systems 
(9) and (12) therefore converge or diverge simultaneously, and 
consequently any sufficient condition for convergence of the 
iterative process for system (12) will also be a sufficient condition of 
convergence for system (9). 
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Thus if one can find positive numbers p,,..., £, such that one 
of the conditions 


1) 5 Ja,| i <1 fort=1,...,2” 
jal pi 


(13) 2) > laud SE < | forj=l,...,n 
2 Gib; 
) 2a <h 


is satisfied, the process of iteration for system (9) will converge. 
Observation. In the practical computation of the iterations we 
may proceed by two methods: 


1) Put X=0. Then 
XW = (I4A4 +++ +Ab1)F = F+AF+ ++ + ARIE. 


For the computation of X‘*) we calculate successively the vectors F, 

AF, ..., A‘-\F and find their sum. This is convenient, because of 

the uniformity of the process of computation, and also because each 

succeeding term is only a correction to the sum of the preceding 

ones. The shortcoming of this method is the possible accumulation 

of rounding errors with the increase in the number of summands. 
2) The computation is conducted directly by the formulas 


X®) = AXG-DLF. 


Here each approximation is treated as a new initial one, and 
there is therefore no need in the first steps of the process to carry 
out the computations to any great accuracy; the errors that arise 
even themselves out later on. 

As an example let us find the solution of the system 


0.78x, —0.02x. —0.12x3—0.14x, = 0.76 
—0.02x, + 0.86x, —0.04x5 +0.06x, = 0.08 
—0.12x,; —0.04x0 +0.72x5 —0.08x, = 1.12 
—0.14x; +0.06x, —0.08x3 +0.74x, = 0.68. 


(14) 
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Solving this system by the single-division scheme, we find 


x; = 1.534965 
xy = 0.122010 
x3 = 1.975156 
x, = 1.412955. 


For the application of the iterative process we bring the system 
into the form X=AX+F.  Itis possible to do this thus, for instance: 


x) = 0.22x, + 0.02%, +0.12x3+0.14x,+0.76 
Xo = 0.02x, + 0.14%. + 0.04x3 — 0.06x, + 0.08 
xg = 0.12%, +0.04%9 +0.28x3 + 0.08%, + 1.12 
x, = 0.14x, —0.06x. + 0.08x3 + 0.26x, + 0.68. 


(15) 


It is readily apparent that the sufficient conditions for convergence 
of the process of iteration are fulfilled. 

Let us compute the successive approximations by three methods, 
for a comparison of the course of the iterative process in the diffcrent 
variants, viz. : 


1) The computation of the successive approximations is effected 
k 
by the formula X) = > A!F (see Table XIII). 
150 


2) The computation of the successive approximations is effected 
by the formula X® = AX 4-0 +F with X =F (see Table XIV). 

3) Recompute X®=AXG%-1+F with X© =(1,0,0,0) (see 
Table XV). 

Table XIII will be explained. The first part of the table contains 
the components of the successively computed vectors A/F. The last 


column is a check; in it are tabulated the numbers > 6, x) , where 
=> a;; (these numbers must be calculated beforehand and 
ee in the same sheet as that on which the elements of the 


nt n 
matrix A are tabulated). Butit is obvious that > dx) = > x(t), 
j=l jal 
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TaBLE XIII. Iterative Method ; Computation 
& 
by the Formula X) = 5 AIF 
iaQ 
l) AIF, /=0,1,...,14 
F 0.76 0.08 1.12 0.68 


xue) 
xas) 


xa 


0.22 
0.02 


0.12 
0.14 


0.50 


0.76 

0.3984 
0.1952 6400 
0.0942 1056 


0.0452 7913 
0.0217 4095 


0.0104 3649 
0.0050 0961 


0.0024 0463 
0.0011 5422 


0.0005 5403 
0.0002 6593 
0.0001 2765 
0.0000 6127 
0.0000 2941 


0.02 
0.14 


0.04 
- 0.06 


0.14 


0.08 

0.030+4 
0.0086 4000 
0.0022 3488 
0.0005 5572 
0.000] 3570 


0.0000 3285 
0.0000 0792 


0.0000 0190 
0.0000 0046 


0.0000 0011 
0.0000 0003 
0.0000 0001 


0.12 
0.04 


0.28 
0.08 


0.52 


1.12 

0.4624 
0.2079 3600 
0.0969 2928 
0.0458 9292 
0.0218 8361 


0.0104 7017 
0.0050 1763 


0.0024 0654 
0.0011 5468 


0.0005 5414 
0.0002 6596 
0.0001 2765 
0.0000 6127 
0.0000 2941 


0.14 
0.06 


0.08 
0.26 


0.42 


0.68 

0.3680 
0.1866 2400 
0.0919 7568 
0.0447 2340 
0.0216 0525 


0.0104 0364 
0.0050 0170 


0.0024 0272 
0.0011 5376 


0.0005 5392 
0.0002 659) 
0.0001 2764 
0.0000 6127 
0.0000 2941 


k 
2) X= > AIF, k= 12,13, 14. 
=0 


1.5348 4720 
1.5349 0847 
1.5349 3788 


0.1220 0958 
0.1220 0958 
0.1220 0958 


1.9750 3858 
1.9750 9985 
1.9751 2926 


1.4128 3762 
1.4128 9889 
1.4129 2830 


1.2592 

0.5984 6400 
0.2853 5040 
0.1364 5117 
0.0653 6551 


(5) 

0.0313 4316 
0.0150 3686 
(79) 

0.0072 1580 
0.0034 6312 


(20) 
0.0016 6219 


0.0007 9783 
0.0003 8295 
0.0001 8381 
0.0000 8823 
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so that the elements of the check column are also equal to the sum 
of the rest of the elements lying in the same row. ‘The second part 
of the table gives the approximate solution of the system, which we 
obtain by summing up the corresponding components of the com- 
puted vectors. 

From a comparison of Tables XIII, XIV and XV with the result 
obtained by the single-division scheme, we see that the convergence 


TABLE XIV, = Iterative Method: Computation by 
the Formula X®) =AX%-V4F; X®) =F. 


0.22 0.14 
0.02 — 0.06 


= (0) 
xa 
X(2) 
x@) 
Xt) 
xis) 
X(6) 
xX) 
Xe) 
X (8) 
X(10) 
xa) 


xu) 


0.12 
0.14 
0.50 
0.76 
1.1584 
1.3537 
1.4479 
1.4932 
1.5149 
1.5253 
1.5303 
1.53273 
1.53389 
1.53445 
1.53472 
1.53485 


0.14 
0.08 
0.1104 
0.1190 
0.1213 
0.1218 
0.1220 
0.1220 
0.1220 
0.12201 
0.12201 
0.1220! 
0.12201 
0.12201 


1.12 
1.5824 
1.7903 
1.8873 
1.9332 
1.9551 
1.9655 
1.9705 
1.97292 
1.97408 
1.97464 
1.97491 
1.97504 


0.08 
0.26 
0.42 
0.68 
1.0480 
1.2346 
1.3266 
1.3713 
1.3929 
1.4033 
1.4083 
1.41072 
1.41188 
1.41244 
1.41271 
1.41284 


3.03838 
(6) 
3.04187 


5.04354 
(5) 
5.04434 


(4) 
5.04473 


(l) 
5.044920 
5.0450069 


X93 | 1.534910 
X04) | 15349385 


0.122010 
0.1220096 


1.975101 
1.9751299 


1.412900 
1.4129289 
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of the process in all three variants is nearly identical: the fourteenth 
approximation gives, in the example in hand, a valid result with 
accuracy to a unit of the fourth decimal place. 

The convergence of the iterative process can be much improved 
by means of a simple device suggested by L. A. Liusternik [1]. 


TaBLe XV. Iterative Method: Computation by 
the Formula X®) = AX¢-N)+F; XO) =(], 0, 0, 0) 


0.98 
1.2412 
1.3912 
1.4656 
1.5016 
1.5190 
1.5273 
1.53129 
1.53320 
1.53412 
1.53456 


0.10 
0.1140 
0.1195 
0.1213 
0.1218 
0.1220 
0.1220 
0.12201 
0.12201 
0.12201 
0.12201 


1.24 
1.6544 
1.8266 
1.9049 
1.9416 
1,9591 
1.9675 
1.97148 
1.97339 
1.97431 
1.97475 


0.82 
1.1236 
1.2714 
1.3443 
1.3798 
1.3970 
1.4053 
1.40928 
1.41119 
1.41211 
1.41255 


1.974962 
1.975063 
1.975112 


1.412761 
1.412862 
1.412911 


0.122010 
0.122010 
0.122010 


1.534770 
1.534872 
1.534920 
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In view of the fact that this device is connected with the approxi- 
mate computation of the first proper number of the matrix, we post- 
pone consideration of it until § 35, in Chapter IIT. 


§18. THE PREPARATORY CONVERSION OF A 
SYSTEM OF LINEAR EQUATIONS INTO 
FORM SUITABLE FOR THE METHOD OF 
ITERATION 


In the preceding section we have established a number of 
sufficient conditions for the convergence of the process of iteration, 
as applied to systems of equations given in the form 


(1) X = AX+F 
or, what is the same thing, 
(I-—A)X = F. 


All those criteria required that the matrix A be small in one sense 
or another, i.e., that in the matrix (/—A) the diagonal elements be 
sufficiently preponderant and be close to unity. 

Let us consider some devices for transforming a given system 
BX =G into form (1) with a suitable matrix A. 

A system 


(2) BX =G 


is especially easily transformed into the necessary form in case the 
diagonal elements of the matrix B considerably preponderate over 
the remaining elements. 

Let us rewrite system (2) in extended form: 


by\X1 + byoXo+ +++ +61,%, = 21 
bo1X) + byoxe + +++ +ba,%, = Be 


(3) 


ee @ @ @ @ @  @ @ 6 @ oe ms# oe @ 
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Divide each equation of system (3) by the diagonal element. We 
obtain the system 


xt pe xet - 4 fla x, = £1 
v1 1 
2) yt t os tote x, = £2 
(4) bee boo bye 
b bi» 
ott xi t Fe tet - +s, = 2 
an mn nn 


This system, in matrix form, is given notationally by 


(5) X = AX+F, 
where 

0 _ d12 bin ay 
by, by, 61) 
_ ber O°. Ses Pon 22 
(6) A = bee boo ’ F = bao 
5,1 bie Bn 
Ban Bae . Dr 


For application of the process of iteration there is no need to 
actually make the transformation of (3) into system (5). The suc- 
cessive approximations may be computed by the formulas 


CP ree eae eta a tg A 2 SO ee 
bx =g—b we Ye ; eb EOD, 
The modification of the iterative process just described bears the 


appellation method of simple iteration. 
The sufficient conditions of convergence of the iterative process 


Preparatory Conversion of Linear Equations 129 


that were deduced in § 17, on being applicd to system (5), give the 
following sufficient conditions for the method of simple iteration: 


1 fl ct 

jor | Oi 

b; 
(8) IT. i < ] 
Il. (Ge i) <1. 

oe 


Here the prime sign on the sum signifies that the value :=y) is 
omitted in the summation. 
If me makes use of the generalized criteria (13) § 17, having put 


p; = | . 1b. we shall then obtain the following additional sufficient 


tokens of the convergence of the simple iteration: 


$l) ca 
jai | ;; 
‘ ’ es b;; 
(8’) W. S$ | <1 
ii | 9); 
F) ay b..\2 
Wr. > si ai. 
jet \O;; 


The transformation of system (2) into system (5) just mentioned is 
obviously equivalent to the multiplication of system (2) on the left 
by the matrix 


(9) D= 


oC 
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We thereby pass to the system DBX=DGC, which we write in the 
form X= (J—DB)X+ DG, 
Assume now that a system 


BX =G 


has been given, in which a predominance of the principal diagonal 
does not obtain. For application of the iterative process this system 
must be transformed into an equivalent system of the form 


(I—A)X = F, 


with the matrix J—A having a predominant principal diagonal. 
For this it is enough to premultiply both sides of the equation BX =G 
by an auxiliary matrix D sufficiently close to the matrix B-!. 
Indeed, after such a multiplication we will obtain the system 


DBX = DG, 


in which the matrix DB is close to B-!B=/. Let DB=J—A, 
DG=F._ Since the matrix A will be small here, this transformation 
will bring the given system into the required form. 

The selection of the auxiliary matrix D may be accomplished by 
various means, for instance by a rough inversion of the matrix B by 
the Gauss method. In case the predominance of the diagonal 
elements in the original system BX =G does exist, but is insufficient, 
it is usually expedient to take as D the matrix inverse to the matrix 


bi; bis 


bo, bee 
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The inversion of such a matrix presents no difficulty, for it reduces 
to the inversion of matrices of the second order, viz.: 


Ay A, 
fa bu 
D = A, A, ) 


where 4, is the determinant 


by) in 


bo) boo 


We note that the necessary and sufficient condition for the con- 
vergence of the iterations in application to the prepared system 


DBX = DG 


consists in all proper numbers of the matrix J/— DB being less than 
unity. 


§ 19. SEIDEL’S METHOD 


Let a system of linear equations be given in the form 
(1) X = AX+F, 


where 4 is the given matrix, F the given vector and X the sought 
vector with components (x),..., %,)- 

The iterative method of Seidel is reminiscent of the ordinary 
iterative process, with this difference, that in computing the kth 
approximation to the component x;, one takes into consideration 
the th approximations, already computed, to the components 
Xy,...,%,;, Explicitly, the computation of the successive approxi- 
mations is performed by the formulas 


i-l n 
(2) =F aja 5 apa Y-4f 
jc =e 


n 
(instead of by x! = > a,x~") with the method of iteration). 
ist 
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In a number of cases it turns out that the Seidel process converges 
faster than the ordinary process of iteration. 
Such is the case, in particular, on condition that 


nt 
All, = se p> lal << 1. 


Indeed, as we have seen, in this case there holds for the method of 
iteration the estimate 


(3) |X—- XH] < pllX— KEV], 
where as the norm of vectors the first norm is taken, i.e., max |x;]. 
i 


We shall show that under this condition the Seidel process con- 
verges, and that a somewhat better estimate obtains for it. 
Indeed, if x,;,..., x, is the solution of system (1), then 


(4) x = > a; x; +f; a=1,...0. 
jel 
Subtracting equation (2) from (4), we obtain 
i-!} Rn 
ae ~ p> aij(x;—x}") = 2 aij)» 
whence it follows that 
i—l n 
(5) all < Slag bal + 3 log Iya? 
Let us employ the symbols 
i—l n 
2 la;;| = B;, 2. {a;.| = Ve 
jel wri 
Then from inequality (5) it follows that 


x; —x)| < B|X-X™||+ y|X-X%-2)]]. 


Taking for i that value ig for which |x; —x"] attains its maximum, 
we obtain 


Vig 
6 X-XW|| < 
(6) |x—x0] <7 


& 
for |X —X || =max |x, —2{9| =[5;.—4%9 1 


|X—XE-D I, 
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Let us use the designation 


Vi ee 
(7) max poe mH. 
Then 
(8) |X-X®|] <p ||X—XE-V |, 
We shall establish that 

(9) HS B. 
Indeed, 

2 la; 5| Bi+¥i 
and 

ees 4 ee B(1—-B; —y;) 
B; +Y; l B l —p; 2 0 


for Bj +y;<n<1. Hence 


pe = max (B;+y;) 2 max eat = pf. 


Here the equality sign is possible only if max > |a;;| is attained for 
s sol 


t= 1, and the reduction of the estimate in comparison with estimate 
(3) will be best if the equations be arranged in increasing order of 


Rn 
the > |a;,|, adopting as the first that equation in which this sum is 
j=l 


least. 

The Seidel method nevertheless does not always prove to have 
the advantage over the ordinary iterative process. 

Sometimes the Seidel process converges more slowly than the 
process of iteration. It is even possible that the Seidel method may 
diverge while the methed of iteration converges. The regions of 
convergence of these two processes are different, overlapping only 
partially. 

We shall now establish the relation between the Seidel method 
and the method of iteration, which will make it possible to indicate 
the necessary and sufficient condition of convergence of the Seidel 
process. 
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Let us represent the equation 


X = AX+F 
in the form 
(10) X = (B+C)X+F, 
where 
(11) 
0 0 0 0 Q11 @yjoe G15 
B= @91 0 0 0 C= 0 aso Q9,, 
an 4,2 aa ae a, n-1 0 0 0 ee ann 


With these symbols one may represent the formulas 
x) = bs aX + > apf +S, 
7 matrix form as 
(12) X®) = BXYW4CXU-D4F. 
Hence it follows that 
Xk) = (1—B)-1CX&-) + (I—B)-IF, 


Thus the Seidel method for system (10) is equivalent to the applica- 
tion of the method of iteration to the system 


(13) X = ([—B)-'CX + (I-—B)-'F, 
which is equivalent to an initial system 
X = (B+C)X+F 


and may be obtained from it by premultiplication by the non- 
singular matrix (J—B)-!. 

From such a representation of the Seidel method it follows that 
for its convergence it is necessary and sufficient that all proper 
numbers of the matrix M = (J—B)-!C be less than unity in modulus. 
These proper numbers are the roots of the equation |M-—Al|=0. 
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Multiplying both sides of the last equation by |J—B| and utilizing 
the theorem concerning the product of the determinants of two 
matrices, we transform the equation into the form 


|C— (I-B)a| = 0, 


or, in extended form, 


Qyy—A aye O12 
(14) AoA Goo —A ore oy - 0. 
a,,A a,2A Ang —A 


Thus for convergence of the Seidel method, it is necessary and sufh- 
cient that all roots of the equation (14) be of modulus léss than 
unity. 

We shall now offer examples exhibiting the difference between the 
regions of convergence of Seidel’s method and the ordinary method 
of iteration. 

Example 1. Let 


5 —5 
= ( 1 O.1 
The proper numbers of the matrix A will then be determined from 
the equation (0.1 —4)(5—A)+5=0, and therefore |A;|>1. The 


ordinary process of iteration diverges. 
Let us form the matrix 


a-syic=(5 49) 


The proper numbers of this matrix are determined from the equation 
A2—0.144+0.5 = 0. 


Obviously |A;|<1. The Seidel process converges. 
Example 2. Let 
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The proper numbers of the matrix A are determined from the 
equation 


~(2.3—2)(2.3+4)+5 = A2-0.29 = 0; |a,J<). 


The ordinary process of iteration convergcs. 
It is readily verified that in this case the Seidel method diverges, 
2.3 —5 
— B)-1C= 
for indeed, (I—B)-C ( ee 
this matrix are of modulus greater than unity. 
If the system of equations is given in the form 


) , and the proper numbers of 


(15) MX = G, 


then in order to apply the Seidel method this system must be sub- 
jected to preparation to make it over into the form 


X = AX+F. 


The method most used leads to a process parallel to the process 
used for simple iteration, viz.: the system 4X =G is tabulated in the 
form 


ro | n 
m-; Ml: : 
x= > i > 7) x; &3 
j=l M;; jortl Mi; mM; 


and the successive approximations are determined by the formulas 


it Rn 
a — _ St my ay Mis e-v 4 Si. 
(16) *j p> m;. %) ee m;; i vn 


From the necessary and sufficient conditions of convergence in the 
general case considered above, those for the convergence of this 
modification of the process may be readily obtained, to wit: for the 
convergence of the process it is necessary and sufficient that all 
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proper numbers of the matrix Q-'R he less than unity in modulus. 
Here 


(17) 
My) 0 0 Myo my, 
Q - Mo, Mase R= 0 0 Mo,, 
Ma, Man vee Map 0 QO 0 


Indeed, the chosen preparation by which the system MX=C is 
transformed into X = AX + Fis equivalent toa premultiplication of the 
system by a diagonal matrix P-!', where P is the matrix composed 
of the elements of the principal diagonal of matrix Af. The matrix 
P-1Q thereby plays the role of the matrix (J—B), and the matrix 
—P-'R the role of the matrix C, and consequently (/—8)-!C 
= —Q-!R; the proper numbers of the matrices —Q-'R and Q-!R 
differ only in sign. 

Furthermore, the characteristic polynomial |— Q-!— Al] of the 
matrix —Q- 2, after multiplication by |— QI], acquires the form 
{(R+AQ|. Consequently for the convergence of the specified modi- 
fication of the Seidel process it is necessary and sufficient that all 
roots of the equation 


m,,A My» feces My 


mo, A Mood eee Ma 


(18) 
m4 mw oce FM A 


be of modulus less than one. 

A great number of sufficient criteria of the convergence of this 
modification of the Scidel method is given ina list by P. A. Nekrasov 
and R. Mechmkce.! 

In particular, criteria I, I, I’, I’ of § 18 are sufficient. 

In case the matrix M of the coefficients of the system ALX= CG is 
symmetric, there exists still] another important sufficient condition 


1R. Mehmke and P. A. Nekrasov [1}, P. A. Nekrasov [I]. 
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of convergence of the modification of Seidel’s process under con- 
sideration, viz.: 

Lf the quadratic form with matrix M is posttive-definite, the Setdel method 
for the system MX=G converges. This condition, on the assumption that 
the diagonal elements of the matrix M are positive, proves to be necessary as 
weill.} 

For the proof we shall put 


(19) M = P+Q+Q, 


where P is the diagonal matrix composed of the diagonal elements 
of the matrix M; Q is a triangular matrix formed by the elements of 
the matrix M that are located below the principal diagonal; and 
Q’ is the matrix of the elements of M located above the principal 
diagonal. On the strength of the symmetry of M, Q’ is the 
transpose of the matrix Q. 

As we have seen above, the necessary and sufficient condition of 
convergence of the Seidel method is the requirement that all proper 
numbers of the matrix T= (P+ Q)—1Q’ be of modulus less than one. 

Let yz; and 4; be any two proper numbers of the matrix T (they 
may, generally speaking, be complex), and let Z; and Z, be the 
proper vectors belonging to them. Then 


(P+ Q)-1Q'Z; = y;Z;, 
whence 


(20) Q'Z; = wjPZ; + 4,QZ,. 


Analogously 
Q'Z; = pyPZ;+ pj QZ;. 

We shall establish a relation of dependence between the scalar 
products (M@Z,, Z;) and (PZ;, Z,;), which will be important to all 
the subsequent discussion. 

To this end let us express (Q'Z;, Z;) and (QZ;, Z;) beforehand 
in terms of (PZ;, Z;). 

We conclude from relation (20) that 
(21) (Q’Z;, Z;) = w(PZ;, Z;) +uj(QZ;, Z;). 

1 E, Reich [1]. 
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Analogously 
(22) (Q'Z;, Z;) = pj(PZ,, Z;) +uf(QZ;, Z,). 
But, on the basis of the properties of the scalar product: 
(Q'Z;, Z;) = (Zp, QZ,) = QZ, Z 
(23) (QZ;, Z;) a (Z;, Q‘Z;) = (Q'Z,, 94, Bs 
(PZ;, Z;) = (Z;, PZ;) = (PZ;, Z;). 
Using these relations, let us write equation (22) in the form 
(QZ;, Z;) = uj(PZ; Z) +n (QZ, Z) 
and, passing to the complex-conjugates, 
(24) (QZ;, Z;) = £(PZ;, Z;) + 4;(Q'Z,, Z;). 
From (21) and (24) we find 
(QZ, Z) = ae (PZ, Z, 
and 
(Q'2;, 2) = BED ‘ HU +A) (pz, Z)). 
We have, morcover, 


(MZ;, Z;) = (PZ,, Z;)+(QZ;, Z;) +(Q'Z;, Z;) 


(25) = (1 +H TED +H) {BO +H) (pz. Z;) 
I—pip; pif; 


a C+ ei) +A)) (PZ,, Z;). 
Hy 
From the last relation the sufficiency of the formulated condition 
for the convergence of Scidel’s method follows directly. 
Indecd, putting :=/, we obtain 


d eee +A) PZ, Z, i), 


1 Hifi = MZ 1» Z;) 
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But 
BA = les (+e) +4) = | +41. 
Thus 
1 +4:,;|2(PZ;, Z;) 
_tje — Lebei ir Z 
for in accordance with the property of a positive-definite quadratic 
form, (MZ,, Z;)>0 (see §3, Paragraph 8), just as (PZ;, Z;) >0. 
The last inequality follows from the fact that the diagonal terms of 
a positive-definite quadratic form themselves constitute a positive- 
definite quadratic form. 
Proof of the necessity of the condition is somewhat more complex 
and is based on the following lemma from the theory of Hermitian 
forms. 


Lemma. If 


n 
P(x, Xo, -++,%_) = ; 2 C5 5% j%; 
t.3> 


is a positive-definite Hermitian form, a, @,...,¢, are arbitrary 
complex numbers and 4), fo, ..., ##, complex numbers less than of 
unit modulus, then 


26 Wixi, te .--,%) = 3 Gy, ete we 
(26) (1) #2 *n) Pa iT," 
is also a positive-definite Hermitian form. 

Proof. Indeed, on the strength of the fact that all |u| <1, 


Pe F ytzet 
l-pAj; 0 BE 


whence 


@ fn 
_ ar 
We > Pe 5S fh ;%;5; 


= 5 D(a, Mir» ofa Xo, ee pix.) >0, 


for all terms of the series in question are positive. 
The lemma is proved. 
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Let us return to the proof of the necessity of the formulated condi- 
tion of convergence of the Seidel method, on the assumption of the 
positiveness of the diagonal elements of the matrix M. 

Let the Seidel process converge. Then all proper numbers x; of 
the matrix (P+ @Q)-1Q’, are of modulus less than unity. Let us 
assume, in addition, that they are all distinct, pair by pair. On 
this assumption the proper vectors Z,, Zo,..., Z, are linearly 
independent and may be taken as a basis of the space. 

Let X be any vector of the space and let x,..., 4+, be its co- 
ordinates on the basis Z,, Zo,..., Z,, that is 


xX = x,Z,+ cee +x,Z,. 
We now introduce into the discussion the quadratic form 


W=(MX, X). In terms of the chosen coordinates it is expressible 
as an Hermitian form: 


7 = : Xx: MZ.,, j= : > oe a 
(2 ) y oe teil i Z;) 2, l—pii; Fad Sed 


(on the basis of equation (25), where ¢;;=(PZ;, Z,)). 
The form 


is positive-definite, on the strength of the assumption of the positive- 
ness of all elements of the diagonal matrix P. 

Accordingly, on the strength of the lemma, the form ¥ is also 
positive-definite, which is what was required to be proved. 

The assumption that was made regarding the absence of multiple 
proper numbers of the matrix T is not essential, for its satisfaction 
can be accomplished by an arbitrarily small deformation of the 
elements of the matrix M. Therefore from considerations of con- 
tinuity , the positiveness of the form (MA, X) turns out to be a 
necessary condition of the convergence of the Seidel methed (on 
the assumption that the diagonal elements of the matrix M are 
positive, of course) even given that multiple proper numbers are 
present. 

We remark in conclusion that the method known as the “ relaxation 
method” for the solution of linear systems is nothing more than an 
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alteration of the computational scheme of the Seidel method. On 
this subject we refer the reader to an article by M. V. Nikolayeva.! 

As an example let us find the solution of system (14) in § 17, 
having brought it into the form 


xX, = 0.22x, +0.02x.+0.12x3 + 0.14%, +0.76 
xo = 0.02x, +0.14x0 +0.04x, — 0.06x, + 0.08 
Xs = 0.12x, +0.04x. +0.28x5 + 0.08x, + 1.12 
x, = 0.14x;—0.06x. +0.08x, + 0.26x, + 0.68. 


The sufficient conditions for the convergence of the Seidel process 
are obviously fulfilled. 


TaBLe XVI. Computation of the Solution of the 
System by Seidel’s: Method 


Sy eg | rp tc | EE | ES 


1.53470 0.12200 1.97494 1.41281 


X10) 1.53486 0.12201 1.97507 j.41290 
XO 1.53492 0.12201 1.97512 1.41293 
X12) 1.534947 0.122009 1.975142 1.412945 
X04) 1.5349579 0.1220094 1.9751507 1.4129513 


1.5349622 0.1220095 1.975154] 1.4129538 


1 M. V. Nikolayeva [I]. 
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As the initial approximation we will take the vector of constant 
terms. The successive approximations are set down in Table XVI. 

Comparing the successive approximations that we have found 
with the solution of the system as found by the single-division 
method (see § 17), we see that the fourteenth approximation gives 
the solution to an accuracy of three units in the sixth place. The 
Seidel process, in the example before us, converges more rapidly 
than the ordinary process of iteration (see Table XIII, XIV and XV 


§ 17). 


§ 20. COMPARISON OF THE METHODS 


In concluding this chapter we will make a comparison of the 
methods described, as regards their application to the problem of 
finding the solution of a non-homogencous linear system. 

The generally accepted criterion of the ‘advantageousness”’ of a 
given computational scheme is the number of necessary computa- 
tional operations, that is, the number of necessary multiplications 
and divisions, and also additions and sultractions. Fundamental 
to this is the number of multiplications and divisions, and therefore 
in characterizing a method from this point of view we have limited 
ourselves to reckoning this number only. These criteria may not, 
however, be the only ones. Simplicity and uniformity of the 
operations to be performed in conformity with a given scheme may 
compel the computor to prefer it to a scheme with a considerably 
lesser number of operations but of more complicated ‘pattern’’. 
The latter circumstance is particularly important in case the com- 
putor has a digital calculator at his disposal. 

Furthermore, one must take into consideration the possibility of 
a compact recording of the intermediate results, i.c., the possibility 
of carrying through the computations by accumulation formulas. 
Compact schemes, although they require higher qualifications on 
the part of the computor, frequently much simplify the entire process 
of computation. An impartant factor influencing the choice of a 
computational scheme is the absence of a loss of significant figures in 
the process of computation by a givenscheme. Finally, the decisive 
factor is the reliability of the results obtained. 
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Let-us analyse the methods we have described from the point of 
view of the factors indicated. 

As was remarked carlier, the iterative methods offer greater 
simplicity in respect of the structure of their computation schemes. 
The separate opcrations are also carried out by accumulative means, 
and the computation can be carried through by a self-correcting 
process. Here it is easy to determine the number of operations 
necessary to obtain one iteration; to determine, however, the num- 
ber & of necessary iterations is ordinarily possible only in the course 
of the work; it may prove to be very large. 

Therefore in case the form of the system’s coefficient matrix per- 
mits one to have confidence that the convergence of the iterative 
process will be sufficiently rapid (say k<n, where n is the order of 
the matrix of the system), this method may well be preferred to all 
others. Together with it one should without fail, of course, utilize 
the device that we analyse 1n § 35 for improving the convergence of 
the process. 

Systems, however, exist for which the generally adopted iterative 
methods either converge too slowly or even diverge. In this case 
only the exact methods make possible the computation of the 
solution. 

Modifications of the most powerful of the exact methods—the 
Gauss method—are the compact schemes, those of the type of the 
single-division scheme, and the elimination scheme. The compact 
schemes of the Gauss method, and in particular the square-root 
method for systems with symmetric matrix, can be recommended 
in case highly qualified computors are available. The sinyle- 
division scheme may sometimes be successfully rivalled by the 
elimination scheme, in which the whole process of computations is 
conducted uniformly, thanks to the absence of a return course. 

The number of operations by the different variants of the Gauss 
method is approximately the same; only the number of results to be 
recorded are different. As has already been remarked in describing 
the Gauss method, its basic defect is the possible disappearance of 
significant figures in case either the determinant of the system itself 
or that of one of the intermediate minors is small. | 

In such cases the escalator method is reliable (we have described 
it only for symmetric systems). 
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Finally, the determination of the solution by means of the inverse 
matrix, theoretically the simplest, is in practice sometimes also the 
most efficient method. 

This will be the case, for example, when the solution must be 
determined with a high degree of accuracy. Indeed, as we saw in 
§ 13, the elements of the inverse matrix can be determined as 
exactly as one wishes, as soon as onc has found a sufficiently exact 
approximation, from which an iterative process giving these elements 
converges extremcly rapidly. For computing the first approxima- 
tion to the inverse matrix, along with the method of Gauss, the 
method of bordering may also be employed with success. 


CHAPTER III 


THE PROPER NUMBERS AND PROPER 
VECTORS OF A MATRIX 


This chapter is devoted to the problems of finding the proper 
numbers and the proper vectors of a matrix. We recall that what 
are spoken of as the proper numbers of a matrix A are the zeros of 
its characteristic polynomial, i.e., the roots of the equation 


a4,;—A &)9 vee Qin 
4g, Agg— A a2" 
\A—Al] = 
an) Gio «es On—A 


(—1)2[An—p,dAn-1 — -++ —p |] = 0. 


As has already been remarked in § 1, the coefficients p; are, but 
for sign, the sums of all principal :th order minors of the determinant 
of the matrix A. The direct computation of the coefficients p, is 
extremely awkward and requires a huge number of operations. 

The determination of the components of a proper vector requires 
the solution of a system of n homogeneous equations in 2 unknowns; 
in order to compute all the vectors of a matrix, one must solve, 
generally speaking, n systems of the form 


(A—A,1)X, = 0, 


where X; = (x);, X;) - - - » X_;) is the ith proper vector of the matrix A. 
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It is thus perfectly natural that special computational artifices 
that simplify the numerical solution of both the problems before us 
should have made their appearance. As in the preceding chapter, 
we shall distinguish two groups of methods: exact and iterative. 
The exact methods, when applied to the first problem, will give 
more or less convenient schemes for determining the coefficients p,. 
The proper numbers will then be obtained as the solutions of an 
algebraic equation of the ath degree. We note that all of the 
methods proposed below, with the exception of that of Leverrier 
(1840), have appeared in the thirties of our century or later. The 
majority of the methods proposed are adapted to the solution of the 
first problem. We shall show, however, that by utilizing the inter- 
mediate results of the computations, the proper vectors of the matrix 
belonging to the corresponding proper numbers can also be com- 
puted after the latter (avoiding the solution of the systems indicated 
above). Only one method—namely that known as the escalator 
method—solves both problems simultaneously, but it makes neces- 
sary the computation, in the course of the process, of the complete 
spectrum of the matrix, and all the proper vectors not only of the 
given matrix but of its transpose as well.! 

The iterative methods make possible the direct determination of 
the proper numbers of the matrix, without resorting to the char- 
acteristic polynomial. In using them, only the first proper numbers 
are as a rule determined with sufficient accuracy. The course of 
the iterative process depends essentially on the structure of the 
Jordan canonical form that is connected with the given matrix and 
on whether the proper numbers of the matrix are real or imaginary. 


2 We note that the determination of the roots of the characteristic equation may 
be efficiently carried out by Newton’s method, viz.: if 


p(A) = AR—p,AA-1— .-- —p, = 0 


is the characteristic equation, then 4,;=J)— , where dp is some initial 


Y (Ao) 
approximation to a root, and A, gives a significantly better approximation than Apo. 
The values of p(Ap) are conveniently calculated by Horner's method: p(Ag) = gn, 
where the numbers g, (¢=0,..., 2) are computed by the recurrence formulas 
G = M-r4o— Pe» 9-1 = 0. 
Analogously 
P' (Ao) = fa-1, where m= 4 yAot a, r-1 = 0. 
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The convergence of the iterative process is determined by the mag- 
nitude of the ratio of the moduli of different, neighboring proper 
numbers, and may be very slow. We shall exhibit several artifices 
making it possible to much improve the convergence of the process. 
A proper vector is computed simultaneously with the proper number 
to which it belongs. As in the solution of Jinear systems, the chief 
merit of the iterative processes consists in the simplicity and uni- - 
formity of the operations to be performed. 

In all the following sections we shall assume the coefficients of 
the matrix to be real. 


§21. THE METHOD OF A. N. KRYLOV 


A. N. Krylov’s work [1] was the first in a great cycle of works 
devoted to bringing the secular equation into polynomial form. 
The idea of A. N. Krylov consists in making a preliminary trans- 
formation of the equation 


@,—-A ayn tise as, 
D oA eee 4 
(1) D(a) a ay agg Qe, ae 0 
anal 4,9 Site fe Qin—A 


into an equation of the form 


baad: Bs cee “be 


° = 2 oo eee 
(2) Digs | ee Ve Se Pe 


bia" bo «1. On, 


which is, generally speaking, equivalent to it, and whose expansion 
in powers of A is obviously accomplished considerably more simply by 
expanding this determinant by minors of the first column. 

In order to effect the transformation indicated, A. N. Krylov 
introduces into the discussion a differential equation, which is con- 
nected with the given matrix; he simultaneously poses the problem 
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of finding a purely algebraic transformation turning equation (1) 
into equation (2). 

Works of N. N. Luzin [1], [2], I. N. Khlodovsky [1], F. R. Gant- 
makher [1], and D. K. Faddeev [2] are devoted to explaining the 
algebraic essence of the transformation of A. N. Krylov’s. 

We shall present A. N. Krylov’s method in an algebraic inter- 
pretation. 

That the determinant 


@3,—A ay9 eos a1, 
(3) D(a) zs fo} fog — A ae Bon 
Gal Qo «ss Gp—A 


equals zero is the necessary and sufficient condition that the system 
of homogeneous equations 


Ax, = 441%, +4,oX%o+ °° + 41,%, 


(4) Axtg = Gq)X1+4qoX%e+ +++ +40,%, 


Ax, = 4y1%1 + G,9X%e+ + °> + 2,,%p 


have a solution x), %o,..., *, different from zero. 

Let us transform system (4) in the following manner. We shall 
multiply the first equation by 4 and replace Ax,,..., Ax, by their 
expressions (4) in terms of x), ..., %,. 


This gives 
(5) A2x, = boix t+ baoret +++ + bank ys 
where 
(6) by = = ad Vad o 


Moreover, let us multiply equation (5) by A and again replace 
Ax;, Avo, ..., Ax, by their expressions in terms of x;,...,%,. We 
shall obtain 


A8x, = b3)x1 + bgoret+ +++ +dagXy 
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Repeating this process (n—1) times, we pass from system (4) to 
the system 


Ax, = byyx%y + byo%at+ +++ +O 4%, 


(7) A2x) = b93%1 + booxat +++ + ban%q 


Anxy = b3%4 + byox0+ aie + BanXns 


whose coefficients 6,, will be determined by the recurrence formulas 


b= > by Qn 5 
(8) sk 2, i-1, k P=) igh 
k=1],...,2” 
big = ayy. 

The determinant of system (7) will obviously have form (2). 
System (7) has a non-zero solution for all values of A satisfying the 
equation D(A)=0. Thus D,(A) reduces to zero for all A that are 

roots of the equation D(A) =0. 


We shall show that 


(9) way 7 a -N, 


eo @  @ #8  @ @  @@®  @ @  @ @ @ 


b,-1,1 On-1,2 +--+ 3 


that is, for V0, D,(A) differs from the sought characteristic poly- 
nomial by a numerical factor only. 

Let all roots of D(A) be distinct. Since all roots of D(A) are roots 
of D,(A), D,(A) is divisible by D(A). Since, in addition, the degrees 
of D,(A) and D(A) are alike, the quotient must be a constant (does not 
depend on 4). Comparing the coefficients of A", we obtain 


D,(A) _ 
Dia ~ ™ 


In case D(A) has multiple roots, the equation 


(10) D,(A) = ND(A) 
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is preserved, which follows, however, from considerations of con- 
tinuity. 

This equation may also be verified directly by a multiplication of 
the determinants figuring in it if relations (8) are used. 

It is evident from equation (10) that if N=0, D,(A) is identically 
equal to zero. In this case the indicated transformation yields 
nothing. 

However even for N=0, A. N. Krylov proposes a special method 
whose algebraic substance will be elucidated below. 

Let us now turn to the coefficients 5,, defining D,(4). We 
introduce into the discussion the vectors B; with components 
(b:3, 6;2,...,9,,). The equations 


(8) bit = 2 Pint, Mat t=2, ae 
Keds 
by, = ai, 
show that 
(1 1) B, = A’B,_), 


A’ being the transpose of the given matrix. 
From equation (11) it follows that 


B, = A'\-1B,, 1=2,...,0. 
In its turn B, =A’Bo, where By=(1,0,...,0). Thus, finally, 
(12) B, = A'iBy, t=1,2,...,0. 

It is obvious that system (4) may be transformed by starting, for 
example, from the second equation of that system. In this case A 
will be put into the second column of the determinant D,(A), and 
the coefficients 5. will be determined by formulas (12), where 
Bo=(0, 1, 0,..., 0). 

The method of A. N. Krylov is quite naturally generalized if we 
introduce into the discussion instead of the vector Bg of special form, 


the arbitrary vector By= (bo), bos, - - » » bon): 
Let 


(13) U = box, + booxet °° + +b ggX qs 


where (x), X2,...,; %,) 1s the solution of system (4). 


f-=\ Oe Dow u 
=n bis bin [x \ sé 
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Then on repeating the preceding argument, we will obtain 


i -_ bas ban 


Uo = boxy +booret +++ +bo0%, 

Au = byx\+byoxet +++ +44,%, 

(14) A2u = bay; +booret+ ++ +> +ba,X, 
Aru as 51%) + b,0%9 + seis +b, Xq) 


where B, =(0;), 5;0,...; b,.) = A%By. 

Regarding the n+1 equations (14) as a system of homogeneous 
linear cquations in n+] unknowns ug, x,,...,x,, we will have a 
non-zero solution possible in case, and only in case, the determinant 


(15) 


Repeating the previous reasoning, we find that 


D(A) = D(a)N, 


where this time 


(16) N= 


b,-1,1 5,-1,2 ++» 24 


Just as for the particular case considered above, the transforma- 
tion yields nothing if N=0. 

Let us assume at the outset, therefore, that N40. On the basis 
of the equation D,(A) = ND(A), the coefficients p, of the character- 


(= 1-1, 
N 


istic polynomial are to be determined as the ratios :, where 


N; are the algebraic complements of the elements 4*-‘ in the deter- 
minant D,(d). And the determination of the coefficients of the 
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characteristic polynomial by means of the ratios indicated indeed 
constitutes the essence of A. N. Krylov’s work. However, the 
investigation there conducted makes possible the determination of 
the sought-for coefficients while avoiding the computation of the 
minors, thus much reducing the number of operations necessary. 
Indeed, in view of the fact that the rows of determinant (16) are 
the components of the vectors Bg, B,,..., B,-3, the condition N40 
is equivalent to the linear independence of these vectors. Therefore 
for NO, the vectors Bo, B,,..., B,-; form a basis of the space. 
Accordingly the vector B, is a linear combination of them: 


(17) B, = 18,-1+ ne +49,Bo- 


We shall show that the coefficients of this dependence are indeed the 
coefficients p, of the characteristic polynomial, the latter being 
written in the form 


D() = (—1)[Ar— pet = +++ =p). 


Indeed, taking from the last row of the determinant D,(A) a linear 
combination of the preceding rows with respective coefficients 
91s Ja» +++» Gq» WE Obtain, on the basis of equation (17), 


Aa-t 
“a <— | An—qyAr-i—- ..- -@, 0 pe 0 
NS tT 
4 = (—1)*{Ar—gqyan-1— --- —g,]N. 


D(A) = PW) = (a ysfar— quae «+ - 9), 


which is what was required to be proved. 

Equation (17) permits the finding of the coefficients ¢,;=/, 
Jo=fo,-.+»9,=f, as the solutions of a system of linear equations 
equivalent to this vector equation. 

Equation (17) connects the method of A. N. Krylov with the 
Cayley-Hamilton relation (applied to the matrix 4’). 
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Indeed, from the relation 


Ar = pyA’a-l + Steed +p,/ 
it follows that 
A™Bo = pA Bot --- +P,Ba; 
that is, that 


(17) B, = p:B,-1+ CES +p,Bo. 


In place of system (17), one can obviously employ for the deter- 
mination of the coefficients p, the system 


(17°) Cy = pPiCa-rt +++ +p,Co; 


where the vectors C,, are determined by the equations C, = AtCp. 

For the determination of the coefficients p; by solving system (17) 
or (17°), one must carry out 322(2+ 1) multiplications and divisions. 
In its original form the method of A. N. Krylov required 
3(n4 + 4n3 4-2n? —n—3) multiplications and divisions. 

In case N=0, the system equivalent to cquation (17) does not 
make possible the determination of the coefficients of the character- 
istic polynomial, since the determinant of this system is likewise 
equal to N. 

The algebraic essence of the device of A. N. Krylov’s alluded to 
above consists in this: it is possible to determine the coefficients of 
the polynomial of least degree, (A), such that ¥/(4)Cg=0. Gener- 
ally speaking, this will be the minimum polynomial of the matrix, 
and its roots will coincide with all the roots of the characteristic 
polynomial, but will be of lower multiplicity. However, with an 
unfortunate choice of the vector Co, instead of the minimum poly- 
nomial some once of its divisors may be obtained, whereupon part 
of the roots of the equation |4—A/Z|=0 may be Jost. As has been 
shown by N. N. Luzin and I. N. Khlodovsky,! with a special choice 
of the vector Cg one can obtain as the polynomial ¥/(A) any divisor 
of the minimum polynomial. 

We note that if the minimum polynomial of the matrix does not 
coincide with the characteristic polynomial, N=0 for any choice of 


IN. N. Luzin (1J, [2]; 1. N. Khlodovsky [I]; F. R. Gantmakher {1}; D. K. 
Faddeev [2]. 
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the vector Co, for ¥(A)Cy=0, and since the degrec of the poly- 
nomial ¥(A) is less than n, the vectors Co, ACo,..., A"-1Coq are 
linearly dependent. As regards an additional degeneration, this 
may be avoided by changing the initial vector Co. 

Thus the method of A. N. Krylov makes possible the determina- 
tion of the coefficients of the characteristic polynomial if V0, or 
of some one of its divisors—generally speaking the minimum 
polynomial—if N=0. 

In practice the circumstance that N=0 will reveal itself during 
the forward course of the solution of system (17) by the Gauss 
method. For in part of the equations all coefficients will be 
simultaneously eliminated, so that these equations will reduce to the 
identity 0=0. These equations (let them be n—m in number) 
must be discarded; in the system that remains it will be necessary 
to discard n—®m of the last columns, beginning with the column of 
constant terms (i.e., with the components of the vector C,,). The 
last of the remaining columns, composed of the components of the 
vector C_, must be taken as the constant member of the new system. 
The solution of the system gives the coefficients of the linear depen- 
dence of C,, on Co, Cy,...;C,-1, that is, the coefficients of some 
divisor only of the characteristic polynomial. 

We shall here analyse two examples, both taken from the article 
of A. N. Krylov’s [1]. 

As the first example let us determine the coefficients of the char- 
acteristic polynomial of the matrix 


— 5.509882 1.870086 0.422908 0.008814 
0.287865 —11.811654 5.711900 0.058717 
0.049099 4.308033 — 12.970687 0.229326 
0.006235 0.26985 1 1.397369 —17.596207 


which A. N. Krylov took from Leverrier’s work [1]. By Leverrier’s 
computation, 


(A) = A+ +47.888430A3 +- 797.278922 + 5349.457A + 12296.555 = 0; 
4, = —17.86303; 4, = —17.15266; As = —7.57404, 
44 = —5,29870. 


ll 
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In Table I we exhibit the scheme for the determination of the 
coefficients p;, regarding them as solutions of system (17'). 

There, in rows 1 to 4, are arrayed the components of the vectors 
AKC, (k=0, 1, 2, 3,4) as they are computed, and also the usual check 
sums. (For computational convenience we tabulate the matrix A 
on a separate sheet in transposed form, and obtain the components 
of the vector being computed as the sum of the products of the ele- 
ments of the corresponding column of the tabulated matrix and the 
components of the preceding vector, in one accumulation.) The 
fifth row effects the check, which is analogous to that employed in 
the example of § 17. 


Taste I. Computation of the Coefficients of the Characteristic 
Equation by the Method of A. N. Krylov. Leverrier’s Matrix 


0 C; Co C3 Cy > 
1 |~5.509882 | 30.917951 |— 179.0125! 1100.7201 948.1156 
0} 0.287865 | —4.705449 | 66.38829 | —967.5973 | —905.62659 
0| 0.049099 | 0.334184 | —23.08728 576.5226 553.81860 
0| 0.006235 | 0.002224 | —0.649152 ~ 4.04003 ~ 4.68072 
_s.166683 | 26548910 |~136.3606 705.6054 
Ty ~16,34603 | 230.62300 | —3361.2885 | —3146.0115 
7 1.136758 | —34.41064 741.5585 708.28462. 
0.104141 | —2,087086 16.91760 14.9346 
! — 30.27086 652.3451 623.0742 
1.065352 | —51.01827|  —49.95051 
~ 47.8887 _ 46.8888 
! | _ 797.287 _ 796.284 
— 5349.53 5348.50 


(8) 
— 12296.8 — 12295.7 


158 Proper Numbers and Proper Vectors of a Matrix 


In rows 6 to 14 is contained the solution of the system obtained ; 
we find this solution by the single-division scheme. 

As a final check on the computation of the coefficients p; we have 
a comparison of the value of $, with the trace of the matrix. Since 
tr A= —47.888430, we see that the value of p, found from the solu- 
tion of the system is sufficiently exact. 

A comparison with Leverrier’s data shows, however, that the 
coefficients have been computed with a lesser degree of accuracy. 
This loss of accuracy is well-known, and is the inherent shortcoming 
of the A. N. Krylov method; it is to be explained by the circum- 
stance that the coefficients of the system determining the f; are 
quantities of different orders of magnitude. 

A somewhat better result can be attained by employing the 
pivotal condensation scheme for the solution of the system. 

As the second example we will take yet another matrix from the 
article of A. N. Krylov’s [1]: 


5 30 —48 
(: 14 -1| 
3 15 —25 


In the subjoined table the first three rows contain the coefficients 
of the system for the determination of the f;: 


The first step of the Gauss process shows that here we have a case 
of degeneration. The truncated system 


M+5q2 = —29 
392 = —]5 
gives as the solution ¢, = —5, gz= —4. Thus in this case we have 


determined the coefficients of the second-degree polynomial 
A2+5A+4, which is but a divisor of the characteristic polynomial. 
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§22. THE DETERMINATION OF PROPER 
VECTORS BY THE METHOD OF A. N. 
KRYLOV 


Let us assume that the coefficients of the characteristic poly- 
nomial have been determined by the method of A. N. Krylov, and 
that all the proper numbers have been calculated and have proved 
to be distinct. We shall show how to determine the proper vectors 
of the matrix by utilizing computations already carried out. Let 
Cy be the initial vector in the A. N. Krylov process, and let 
X,, Xo,..., &, be the proper vectors of the matrix A belonging to 
Ay, Ag,...,4, In §3 it has been shown that X,..., X, are 
linearly independent. 

Let us resolve the vector Cy in terms of the proper vectors: 


(1) Co = aX) +agXot+ +--+ +4,X,. 
We have, moreover, 
Cy = ACg = yy Xy + aydeXy+ +++ +0,A4,X, 
(2), 0 ee eh SS eee as he a ee 
Cy = A™1Cy = a AX, +adg Xa t + +a AP ot X., 
The vectors C,, Co,...,C,.; have been computed in the course of 


finding the proper numbers. Let us form a linear combination of 
them: 


BroCn-1 t+BirC,-et+ + ° +81, p-1Co 
= @(Bi9A, +B At + cee bBy na) X1 


+a9(Bi9Ag 1 +B ds 7+ --- +Bi, ,-1)Xe2 

(3) Sees nko Sheen 
+a,(Bigdn ' +BiAn 7+ +++ +B 1X, 
= 191 (Aq) X, +g (Ag) Xo + pacie +4,91(A,)X,, 


where 


(4) P(A) = Bid 1 + ByArm2 + +) + Bi aa 
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Let us select the coefficients B19, ... , 81, ,-1 so that 
(5) Pilar) #0, gilde) = +++ = gy (A,) = 0. 
To achieve this it is sufficient to take as g,(A) the polynomial 
pila) = (A—Az)... (A—A,) 


(6) wm (AA) (A= Ag) «(A= An) _ (= 2p) 
A—A, A—A, 
Pe sat Pd Cad alia 
A-A, ; 


Here (A) is the characteristic polynomial, whose coefficients and 
roots are already computed. 

The coefficients of the quotient (6) can easily be computed by the 
recurrence formulas 


(7) Bo=1, Aj = AP, ;-1— Pp 


j=l1,...,2—1 (Horner’s scheme), 


which issue directly from the equation 


(A—A))(BigAt1 + BAe + +++ +81, 2-1) 
= (—1)*[As—piAr-1— --- —p,]. 
Thus 
BioC, ~-it+BiiCy-2+ °° +B, a-1Co = &193(A1)X), 


that is to say, the linear combination that we formed is the proper 
vector X;, accurate to a numerical factor. Of course the coefficient 
«, must be different from zero; this is guaranteed by the successful 
completion of the A. N. Krylov process. Since the proper vector is 
determinate but for a constant factor, we may adopt for the proper 
vector the constructed linear combination. 

Analogously 


n-l 
(8) X; = 2 B;C,-1-j» 
where 
(9) Bio = I, Bi; = AB, j-1—- Pj» j=l,...,a—1. 


As an example we shall compute, for the Leverrier matrix, the 
proper vector belonging to the proper number A, = — 5.29870. 
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We reproduce the characteristic equation with the coefficients 
taken from Table /: 


A4 + 47.8887A3 + 797.287 A2 + 5349.534 + 12296.8 = 0. 
Computing the numbers f,; for 7=0, 1, 2, 3, we obtain 
1; 42.5900; 571.615; 2320.7]. 


We form the linear combinations in accordance with formula (8), 
arranging the computations in the table 


Bs3Co Bs aC By C2 | Bs oC | Ae" 


acme | cena ae ES 


2320.71 | — 3149.53 1316.80 | —179.01 308.97 


164.548 — 200.405 66.388 30.531 0.098815 


28.066 14.233 — 23.087 19.212 0.06218) 
3.5640 0.0947 — 0.6492 3.0095 | 0.009740 


The last column contains the components of the proper vector 
X,, normalized so that its first component equals unity. We shall 
see below that the values of the components of the proper vector 
computed by the mcthod of A. N. Krylov agree well with the values 
computed by other methods. 


§ 23. SAMUELSON’S METHOD 


The method suggested by P. Samuelson [1] is close to the A. N. 
Krylov method. 
The computational scheme of this method is as follows. One 
computes the rectangular matrix 


(1) 

if R : 0 8) 0 0 ... O 0 } —~— ay) 
RM :0 0 0 OO. ....0 1 =a, —RS 
RM2 :0 0 0 O ...1 —a,, —RS —-RMS |> 


._ ses e@ e8© @ e@ @ @e¢ oe @ @®  @ ee e® e« oe  e# ee «a 


RMe-1 2 | -—ay, ~RS —RMS....... . . —RMn-2S 
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where R, S, and M are cells in the following partition of the given 
matrix: 


&y\ 4) ain 

411 R 

(2) A= ao, aoe ao, S M 
aal : 4,2 ove ann 


Furthermore, by means of elementary transformations (as this is 
done in the problem of elimination, § 12) one must attain this situa- 
tion: that in place of the row RM*—! a zero row turns up. Then the 
remaining elements of the last row will give, generally speaking, 
the coefficients of the characteristic polynomial. The process of 
elimination, as we have seen, is very uniform and simple. This is 
the fundamental merit of the scheme. 

The author derives his scheme from a transformation of a system 
of linear differential equations connected with the matrix into one 
differential equation of order n, by means of a special device for 
elimination. A brief algebraic rationale of the scheme is contained 
in the following. 


Let X10 
g X90 *10 
(3) Xo = ee aa Y, 
*n0 
be an arbitrary vector. 
Furthermore, let 
*\1 *1, n-1 
X01 “11 x2, n—1 a | a] 
AXo = = ; woe, Arr lyY, = = ; ’ 
: : ( Y, ° ( Y,~1 
(4) Xl . Xn, n-1 
n 
Xon 
ned We 
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From the construction it follows that 


(5) ik 
(6) Y, = Sx, y1tMY,_1 hele cy 


Oy)X1, prt RY, 


Thus we have n? relations (n and n(n—1)) between n2 +n quantities. 
They make possible the climination from the system of equations (5) 
and (6) of the vectors Y;,..., Y,, that is, of a(a—1) quantities. 
As the result of this elimination there remain n equations connecting 
2n numbers, these being the components of the vector Yp and the 
numbers x19) «- + 5 Xtp: 

Let us carry out this elimination. We have, fork=1,2,..., a: 


Kyg = OyyXy gi tRY,_1 
441%) yp t RSX, po t+ RMY,_ 2 
443%, pp tRSx) po + RMSx,, yy + RM2Y,_ 5 


24 )X1, gt RSxy yg + RMSX,, 4-3 


+++ +RMt-2Sx, 9+ RM!I-1Y,, 
or 


(7) RME-lY, = ye — 911%}, k- 1~ RSx,. | ee aes — RM-2Sx jo. 


The coefficients of these n equations obviously form matrix (1). 
Eliminating from these n equations the components of the vector 
Yo, we shall obtain a single lincar relation between the numbers 
X10) -- -»)%1, With constant coefficients independent of the choice of 
the initial vector. 
On the other hand, proceeding from the Cayley-Hamilton relation, 
we have 


Xin Pi%), n-1 7 8 °° a Pa*X 10 = 0. 


This equation is also a relation of linear dependence between the 
numbers x39, ... , Xj,, With constant coefficients independent of the 
choice of the vector. 

This relation of dependence will coincide with the relation of 
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dependence obtained by the method of climination in case the 
matrix A is such that we may rightly consider the numbers 
X10) +++» %1, ,-1 to be independent variables, that is, in case we may 
assign them, independently of one another, arbitrary values by 
using a suitable selection of the remaining components of the initial 
vector Xq, or, in other words, by using the vector Yo. 

The Samuelson method may be more rigorously proved by means 
of the following relations between the coefficients of the character- 
istic polynomials of the outer and inner matrices of our bordered 
system. 

Let 


f(A) = (= Mant pdt + +p) 


(8) 
P(A) = (—Va[Aet + gyaer2 + +++ 49,1] 
be the characteristic polynomials of the matrices A and M. (Con- 
travening the usual notational usage for the polynomials, we have 
changed the signs of the coefficients p, and q,.) 
Then the following relations are valid: 


Py 
fo = —RS—91411+ 92 


—41;+49 


(9) bs = —RMS—q\RS— 9203; +93 


for = —RM-8S—q,RMe-4S— +--+ +9,-1 
— RMo-2°S —q,RM-3§ — -- +» —g,_oRS—q,_14)). 


= 
! 


These relations are obtained from the rule for the expansion of a 
bordered determinant. 

Moreover, if the A. N. Krylov method be applied to the matrix 
M, having adopted R’ (with the components ajo, ... , 4),) for the 
initial vector, the coefficients 9, go,..- 5 %,—1 Will then be defined 
by the system of equations 


(10) M's-1R! +-4,M'2-2R' + «++ +9,_,R' = 0. 


1.870086 | 
—20.264529 
261.81060 


— 3882.0495 
l 


Tas_e II. Computation of the Coefficients of the Characteristic 


Equation by Samuelson’s Scheme. Leverrier’s Matrix 


0.422908!  0.008814| 0 | - 0 
| 5.208653| 0.051697| 0 | 0 
| ~183.23653 ~0.905064/ 0} 1 
| 3870.8749 | —10.72247 | 1) 5.509882 
| 0.226144! 0.004713 ‘ 0 
| 9.791355; 0.147204 : 0 
—242.44343 | —2.138977; 0] 1 
4748.777 7.57363 | 1} 5.509882 


0.0150341 0 | 0 


| 4.505918] 0] 1 
—63.81948 | 1 | 5.509882 
l 0| 0.664047 


| 
ae 


797.297 | 5349.60 


0 1 
1 5.509882 
5.509882!  —0.559152 
0.559152 5.577387 
0 0.534735 
! 16.346035 
5.509882| — 140.5584 
—0.559152]}  2081.445 
0.102131 1.669435 
30.270872|  264.18511 
~ 485.5565 | —5846.330 
20.10128 |  175.43127 


5.509882 

— 0.559152 

5.577387 
—66.36373 

2.946325 


59.146736 


— 765.80173 
11371.416 


6.040710) 


698.72872 
— 17314.569 
463.98856 


12296.94 


2 
8.811690 


— 9.053449 
89.197123 
— 76.73268 
4.711917 
86.43 1330 
— 1144.4327 
18215.162 
8.827310 
995.6906 
— 23703.765 
661.185 i. 


(3) 
18492.73 
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On the strength of relations (9), the coefficients p), po, ..., P, 
are non-homogeneous linear forms in q;,..., ¢,-1 and consequently 
may be calculated simultaneously by the method of elimination (see 
§ 12). Of the two possible modifications of the method of elimina- 
tion, one should take that in which the components of the vectors 
R’, M’R’, ..., M's-’R’, are arrayed in the rows of the scheme. Then 
these rows, conceived as matrices, are R, RM,...,RM*-!. The 
coefficients of the relations (9) will thereupon turn out to be arranged 
so as to accord exactly with the Samuelson scheme. Given the 
grounds for the method that have been presented above, the region 
of its application is easily discerned. Indeed, it coincides with the 
region of application of the A. N. Krylov method for the matrix 
M, proceeding from the vector R’. 

As an example we shall again take the Leverrier matrix. We 
carry through the computation of the coefficients of the character- 
istic polynomial in accordance with the scheme described (see 
Table II). At the outset we compute matrix (1), arranging its 
elements in the first four rows. We next perform the elimination as 
was shown in §12. The last row gives the sought values of the 
coefficients, which as regards accuracy almost coincide with the 
values computed by the method of A. N. Krylov. The last column, 
as usual, is the check column. 

The number of operations necessary to determine the coeflicients 
of the characteristic polynomial by the Samuelson method is some- 
what less than by the A. N. Krylov method, for the formation of 
matrix (1) requires n(n—1)? multiplications, and the process of 
elimination in the Samuelson scheme requires just as many opcra- 
tions as does the solution of the system in the A. N. Krylov method. 
In connection with this we comment that in Wayland’s article [1] 
the reckoning of the number of operations for the Samuelson method 
is incorrectly done. 


§ 24. THE METHOD OF A. M. DANILEVSKY 


An elegant and very efficient method of computing the coeffi- 
cients of the characteristic polynomial has been proposed by A. M. 
Danilevsky [1]. The gist of his method consists in an initial reduc- 
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tion of the secular determinant to the form known as the Frobenius 
normal form: 


Pi-A fo pg wes Pa 

] —A 0O 0 
D(a) =| 0 a | 0 |, 

0 0 O ... —a 


the expansion of which in powers of A demands no labor, since 
D(A) = (—1)s[4e—p,Ar-}— «++ — pr). 


On the strength of the fact that similar matrices have identical 
characteristic polynomials, in order to attain the aim set, it is 
sufficient to reduce the given matrix 


411 412 +++ Ain 


yo &91 499 aon 
Qni 42 Gan 
to the form 
pi po ‘ Pr-1 Pr 
1 O 0 O 
P= Oo i! 0 90 


eo @  j@©  @ @ @  e8  @  @® 


0 0 ... I 0 


by means of a similarity transformation. 

We shall show that we can find (2 — 1) similarity transformations, 
the successive performance of which will realize the desired transi- 
tion from matrix A to matrix P, if this is possible. 

Let us examine the beginning of the process. 

We must carry the row 4,1 ago «+. Gq n-1 Qq, into the row 
00... 1 0. Let us assume that a,,,_,;40. Divide all clements 
of the row by a, ,_,; then subtract the (2 —1)th column, multiplied 
by 4,1) 4,9) +++» Qng Fespectively, from all the rest of the columns. 
The desired transformation will thereby obviously have been 
effected. 
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The operations we have indicated will thus be elementary trans- 
formations on columns, and, as has been shown in § | Paragraph 10, 
will reduce to a postmultiplication of the matrix A by a matrix M,-, 
which is not hard to jot down, to wit: 


] 0 Sat 0 0 
0 l 0 0 
M., OO ie ce 
_ Qn) = an° l = aan 
a, a—l a, n-1 Gy, n-1 a, n—1 
0 0 0 ] 


The constructed matrix AM,_, will not be similar to the 
matrix A. However from it one can easily pass to a matrix that is 
similar to A. To accomplish this it is enough to premultiply the 
matrix AM,_, by the matrix M7!,, which is easily computed ; indeed 
one can directly verify that 


1 0 0 0 
| 0 0 
Ye 
Qn1 Ane Gn,n-1 an 
0 QO 0 l 


The multiplication of the matrix AM,_, on the left by the matrix 
M;_', obviously does not change the transformed row. 
Thus the matrix 


C11 Cg we OL, gt Cin 
C21 €29 C2, n-1 Con 
M,;!,4M,_, eed [1M), 2G, cues es 2. Se ck Ao eh ee wee Se, OS SO 
Co~1,2 €n-1,2 °° A-l,n-l  Sn—1, 2 
0 0 ore l 0 


has one satisfactory row, and we can continue the process another 
step further ifc,_,,-2#0. If all the (n—1) transformations are 
possible, we shal] have brought the matrix A into form P. 
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An explanation of how one is to proceed in those exceptional 
cases when the necessary elements vanish will be given below. 
At the moment we shall show how the elements of the matrix C, 


the result of one similarity transformation, are computed. 
We have 


@iy G12 vee Gy nt 41, 
AM,- = 
na-1, 1 a,-1, Rs an-1, n—l Gy_1, n 
an} a,9 a a an, n-1 ann 
l 0 ss bead 0 0 
x 
M,-1, l m,-1, Sy sees Ma-1, n-1 Mm, 1, n 
0 0 bs 0 l 
by b10 by n—1 b1, 
bo boo be n—l bo, 
= = B, 
6-1, 1 b,-1, 2° ees b,-1, te | b,-1, n 
0 0 aes l 0 
where 
big = Aig H4;, g—1My-1, & for allign—1, kAn—-1, 
Bi nd = 45, gy), p12 
Here 
Ma-1.j Oni for t x n— | 
" a, n-1] 
and 
= = ] 
a-l,a-1 ~ Gn nl 


We see that all the elements of the matrix B=AM,_,, with the 
exception of the elements of the (n — 1)th column, are to be computed 
by formulas involving two terms. 

The elements of the (a — 1)th column, if we recall the value of the 
coefficients m,_; ,-1, are obtained as the quotients of the division of 


ma WwW He 
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—> 
My MM, 


0.006235 
0.269851 
1.397369 
— 17.596207 


—-> 
Mi) MM; 


0.185920 

6.046177 

— 29.461961 
— 208.45592 


Miy My 


2.952517 
~ 42.52167 
— 562.5575 
— 2244.455 


l 
— 5.509882 
0.287865 
0.049099 
0.006235 


— 0.004462 


— 5.511769 
0.262379 
0.106974 
0 
0.185920 

— 0.030750 


— 5.566763 
0.659506 
0 
0 
2.952517 
7 + aed 
— 1,685430 
} 
0 
0 
— 47.88843 


2 3 4 2 = 
1.870086 | 0.422908 0.008814 — 3.208074 
— 11811654 5.711900 0.058717 ~ 5.753172 
4.308033 — 12.970687 0.229326 — 8.384229 
0.269851 1.397369 — 17,596207 — 15,922752 
0.193114 0.71563059 12.592384 11.394808 


1.788417 
— 12.914702 
6.812854 
0 
6.046177 


+ 16539377 


0.295793 
— 2.136011 
1 
0 
— 42.32167 
14.33410 


— 79.49874 


~ 797.2789 


8 
0.302646 $.334234 1.91352? 


4.087610 71.985155 63.420442 
9.282220 — 163. 10255 — 165.46494 
J 0 I 
— 29.46196 1 — 208.45592 — 231.68578 
4.872825 34.4773 38.31938 


1.610881 
59,332832 


(6) 
70.74035 
(90) 
— 433.5989 


66.99404 
— 373.2790 
1) 
0 
— 2244.455 
760.1836 


9.017289 
— 58.84347 
0 
J 
~ 562.5575 
190.5349 


70.44456 
~ 431.4629 


— 1051.645 


— 4164.769 


—5295.913 


— $349,455 ~ 12296.55 ~— 18491.17 
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the elements of the (n—1)th column of the matrix A by the clement 
a 


sean 
] 0 0 0 
0 ] 0 0 
C = M7',AM,_, = 
Qn1  Ag2 Qn, n-1 Fan 
0 QO 0 ] 
bi, yg by 1 Oty 
be, bas as 
vd Ce ey ee ae 
b,-1,1 9-1, 2 bn, n-1 On-1, 
0 0 ] 0 
biy bie by nt by, 
boy br» bo, a1 by, 


> Gyber D> Ande «ee Dd Ang, n-t 2 Fnk Pen 
kol kal kal kal 


0 0 Gs I 0 


Thus the premultiplication by M,_', changes only the (xn —1)th row 
of the matrix B; the elements of this row are found with one accumu- 
lation. The entire process, consisting of (n—1) similarity trans- 
formations by means of the matrices M,_,, M,_»,..., My, is fitted 
into a convenient computational scheme, which we shall show in an 
example. 

As the example we again take the Leverricr matrix (Table ITT). 
The Danilevsky transformation consists in the successive multiplica- 
tion of the given matrix on the right by the matrices M3, Mo, and 
M,, and on the left by the matrices Mz ', Mz', M,', where the right 
and left multiplications alternate. Each matrix of the transforma- 
tion is given as one row. 
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Let us pass on to the description of the computational process. 
In rows 1, 2, 3, 4 are arrayed the elements of the given matrix and 
the check sums. We begin the computation with the computation 
of the elements defining the matrices My and M,", viz.: in row 4' 
we enter the elements of the third row of the matrix M3: 


24), &439 l : Ay4 
my) = b ] Ngo — oe 9 m = 9 My =—_ —_ ——e 


A493 &43 aie 449 . G13 

The elements of the third row of the matrix My!, equal to the 
elements a4,(4 =1, 2, 3, 4) we enter, for computational convenience, 
in a column: rows 5, 6, 7, 8 of the head-column I. The result of 
the multiplication by Mg we write in the four rows 5, 6, 7,8. The 
transformation Mz brings the fourth row of the matrix into canonical 
form (the 8th row). The elements of the third column are now 
obtained by multiplying the elements of the third column of the 
matrix A by m33 = — The rest of the elements are computed by 
the two-term formulas: 

(1) big = Gig +A;gMgy. 

For the check we form the column of sums, as usual. The cle- 
ment of the check column located in row 4’ must coincide with the 
sum of the elements of this row after replacing the element m33 by — 1. 
The results of the application of formulas (1) to the elements > are 
entered in the column 5’; adding the elements of the 3rd column to 
them, we obtain the check sums for the rows 5 to 8. 

The transformation Mz! changes only the 7th row; we enter the 
result in the 9th row. Its elements are obtained as the sum of the 
products, by pairs, of the elements situated in column I by the cor- 
responding elements of each column of the matrix AM3. With this 
we have concluded the transformation M,!AM3. The process is 
continued analogously. Weremark that the elements of the 9th row 
are also the elements of the 2nd row of the matrix Mz!; we copy 
them columnwise in column I. The elements of the second row of 
the matrix Af. are entered in row 9’. 

The process is easily learned and is afterwards executed without 
difficulty. 

As the result of the computation we obtain all the coefficients at 
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once, since the coincidence—the check—of p, with the trace of the 
matrix is in addition an index to the accuracy of the computation of 
the rest of the coefficients; the results obtained are closer to the 
Leverrier data than are the results found by A. N. Krylov’s method 
or those by the Samuelson method. 

The number of operations necessary for computation by the A. M. 
Danilevsky method is substantially less than by the two other 
methods referred to above. The number of multiplications and 
divisions is equal to (n—1)(n2+n—1). 

We remark that A. M. Danilevsky’s method admits of a modifica- 
tion similar to the pivotal condensation scheme in connection with 
the Gauss method; it somewhat increases the accuracy of the results 
obtained. 

Knowledge of the matrices M,, Mu, ..., M,_, which successively 
effect the similarity transformations, permits us to determine the 
proper vectors of the matrix A. 

Indeed, let 4 be some proper number of matrix A and 
X=(x,,...,%,) the proper vector corresponding to it. Let 
Y=(y), Yo,--.,¥,) be a proper vector of the matrix P. Then, as 
was shown in § 3 Paragraph 10, 


xX = M,_1M,_2 eee M.M, Y. 


Now, the proper vector of the matrix P is found without trouble, 
since its components will be the solutions of the recurrence system: 


(pi-A)yi t+poyot +++ +P,y, = 0 


Yn-1 — AY = 0. 


This gives y, =1, y,y=4,..., 4, =A-}. 
The transformation M,, performed upon Y, gives 


oo @ $e j@® ®  « 


0 O> ges. A y, UP 
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The transformation M, thus alters only the first component of the 
vector Y. Analogously, the transformation Mg alters the second 
component, etc. 

Thus the components of the vector X are determined by the 
formulas 


k-1 tt 
(2) =D mx,+ 3 my, (k=l... 2-1). 


As an example let us consider, for the Leverrier matrix, the com- 
putation of the proper vector belonging to the proper number Aj. 
From the equation 


A‘ +.47.8884313 + 797.2789A2 + 5349.455A + 12296.55 = 0 
we obtain A,= —5.29872. The computation of the proper vector 


belonging to A, is performed in conformity with the subjoined table. 


TABLE IV. Computation of a Proper Vector by the 
A. M. Danileusky Method 


1.000000 
14.33410 . 0.165394 |— 0.193114 28.0764 10.1446 | 0.098822 


190.5349 4.872825 | 0.715631 — 5.29872 6.38334 | 0.062182 
760.1836 34.4773t | 12.59238 ] | 0.00974) 


Here columns I, II, III contain the elements m),, m,, mg,, which 
we transcribe from rows 14’, 9’ and 4’ of Table ITI, setting them up 
in columns. Column IV contains powers of A,, from the third to 
the zero-th. Column V contains the components of the proper 
vector, which are computed one after another by formula (2), which 
is reminiscent of the formula of the Seidel method. Column VI 
contains the components of the vector X,, normalized so that the 
first of its components is equal to unity. 

In concluding this section we shal] dwell on exceptional cases. 
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Let us assume that after several steps of the process we have 
arrived at a matrix of the form 


C11 C12 Cig a 

C91 992 Co, Co, 

C= fxr x Oke “kn 
0 0 ] 0 0 

0 0 0 I 0 

0 0 ... O OO... 1 =O 


and together with this it turns out that ¢, ,_,=9. 
. Here two cases are conceivable. 

If any one of the elements ¢,, (1<—1) is different from zero, we 
interchange the ith and (f—1)th columns and simultaneously 
change the rows with the same numbers. Such a transformation is 
equivalent to a multiplication left and right by a matrix S (see Para- 
graph 10, § 1) of the form 


l 


Oi assrckewienaee ] t 
: | 
§ = : 
Locus bseawiaecs 0 k—] 
] 


This matrix has the property that S2=/, and, accordingly, 
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S=S-!, so that a multiplication on left and right by S is a similarity 
transformation. After these transformations one must proceed as 
usual. 

If, however, all c,;=0 for :<k—1J, then the matter becomes even 
simpler. 

To wit, in this case the matrix C has the form 


C11 C12 (14-1 1,8 Cin 
Cp-1, 1-1, 2+ + + Cg-1, 4-1 “k-1, Ch-li an 
C, D 
C = 0 0 e004 0 Cu =o lin = 0 C ) 
0 0 0 0 : 
0 0 0 0 l 0 
where 
C1) 5 cee C1, k-1 Che oak Cin 
] Q) 
Ci = > Co = 
Ce-1,1 cee Opa g-1 0. seid. 0 


and consequently 
|C—al| = |C,—al]-|C.—al}. 


The matrix C, already has the Frobenius canonical form and 
therefore |C,—A/| is computed instantly. In order to expand 
|C, — AZ|, one must apply the general process leading to the matrix P). 

Thus the case where the process is broken introduces only 
simplifications into the problem of computing the characteristic 
polynomial: an easily computable factor is separated from the 
characteristic polynomial, and the remaining factor is the charac- 
teristic polynomial of a matrix of lower order. 
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§ 25. LEVERRIER’S METHOD IN D. K. 
FADDEEV’S MODIFICATION 


In this section we shall expound a method known as Leverrier's 
method [2], requiring a greater number of operations than any of 
the methods presented above, but utterly insensitive to the individual 
peculiarities of the matrix, in particular to “gaps” in the inter- 
mediate determinants. 

Let 


(1) D(A) = (—1)*[Ae—pyAr-} — podr-2 — -- - —p,] 


be the characteristic polynomial of the matrix and 4,, A.,..., A, its 
roots, among which some may be equal. Let us employ the symbol 


& 
Ay = Sy. 


i b}Aa 


(2) 
Then a relation is valid that is known as the Newton formula: 
(3) Kop = 84 —fPisgir— es ~fe-isty K=),.. 2m. 


If the numbers s, are known, then by solving the recurrence 
system (3) we can find the coefficients p, which we need. 
We shall show how the numbers s, are determined. We have 


5; = Ay +404 eer +A, = trA. 


Moreover, on the strength of Paragraph 11, § 3, the characteristic 
numbers of the matrix 4 will be At, Af,..., ak. Accordingly 


(4) s, = Abt +--+ At = tr At. 


Thus the process of computation reduces to the successive com- 
putation of the powers of the matrix A, then to the calculation of 
their traces and, finally, to the solution of the recurrence system (3). 
The computation of the n powers of the matrix A (of the last matrix 
AP it is necessary to compute only the diagonal elements) requires a 
great number of operations—uniform, granted—and Leverrier's 
method is inordinately more laborious than the methods expounded 
above. Its value consists, as has already been mentioned, in its 


— 5.509882 
1.870086 
0.422908 
0.008814 


— 232.94165 


— 12296.551 
-0.001 
0.001 


0.001 


0.287865 

— 21.811654 

5.711900 

0.058717 
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0.049089 
4.308033 
— 12,970687 
0.229326 
— 47.888430 


— 427,95884 


— 797,27876 


— 4213.8419 


— 5349.4555 


0.000 

— 0,000 

~ 12296.550 
0.002 

— 12296,551 


0.006235 


0.269851 
1.397369 
— 7.596207 


~- 532.69187 


— 4649.1712 


— 0,000 
0.000 
0.001 


— 12296.551 


$2.378548 
0.287865 
0.049099 
0.006235 


564.33711 
9.07995 
2.68546 
0.3008] 


2258.1433 
70.5604 
32.0618 

4.4283 


— 0.183640 
— 0.005738 
0.002607 
— 0.000360 


1.870086 
36.076776 
4.308033 
0.269851 


58.98700 
396.31360 
99.69550 
11.01857 


458.3882 
1255,.4144 
419.6360 
52.7398 


— 0.037278 
= 0.102095 
— 0.034126 
— 0.004289 


0.422908 
5.711900 
34.917743 
1.397369 


23.13088 
132.18346 
369.31992 

25.74B58 


276.1613 
556.3836 
1135.6136 
98.8129 


— 0.022458 
— 0.045247 
— 0.092352 
— 0.008036 


0.008814 
0.058717 
0.229326 
30.292223 


0.42522 
2.39755 
4.22567 
264.58689 


6.2599 
11.4757 
16.2164 

700.2843 


— 0.000509 
— 0.000933 
— 0.001319 
— 0.056950 


44.68036 
42.13526 
39.50420 
31.96568 


646.8802 
(6) 
539.9745 


475.9266 
(8) 
301.6549 
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universality. The number of multiplications necessary in the 
Leverrier method is equal to 4(n—1)(2n3 — Qn? +n +2). 

We remark that in computing the powers of a matrix it is uscful 
to perform a check by means of a column composed of the sums of 
the elements of each row of the matrix A. The result of the multi- 
plication of the matrix A by this column must coincide with an 
analogous column of the matrix A?, for let >, be the column of 
the sums of matrix A, >» that of the sums of the matrix A?, and let 


Us 
] 
Then: 
>, = AU, So = ALU, 
that is, 
(5) De = AD. 


What has been said is obviously true for the other powers too. 

We now expound the modification of the method that was pro- 
posed by D. K. Faddeev! which, in addition to simplifying the 
computation of the coefficients of the characteristic polynomial, 
permits us to determine the inverse matrix and the proper vectors 
of the matrix. 

Instead of the sequence 4d, A?,..., A", let us compute the 
sequence 4y, Ao,..., A,, constructed in the following manner: 


A, = A, trd, =q,, B, = A,-9\f 


Ap = ABy, “22 = qs, By = As—gul 
(6) 
‘A,-} = AB, _ 2, reat = Wn-1s B,-1 = An-1—4q-il 
A, = AB, 1) ras = ny B, = A,— 4,4: 


1D. K. Faddeev and I. S. Sominsky (1). 
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We shall show that a) 9; =f), do=fo,-+ +5 Gn=Pa3 


b) 8B, is a null matrix; 


c) if A is a nonsingular matrix, then 


(If the matrix A is singular, then (—1)*-!B__, will be the adjoint of 
matrix A.) 
We shall prove a) by the method of mathematical induction. 
That p,=tr A=q, is obvious. We shall assume that 9, =f), q.= fo; 
->%-1=f,-1, and shall prove that g,=p,. In accordance 
with our construction: 


A, = Ak—q,At-1— .- — Gy A 
= At—p,At-l— ... —p, JA. 
Accordingly 
tr A, = kg, = tr At—pytr AA-1— «+. —p, itr A 
= S5p—PiSp_-y— + —Py-151- 


Hence, on the strength of the Newton formula, kq,=kp, 9,=),. 
Furthermore, on the strength of the Cayley-Hamilton relation: 


(7) B, = As—pyAv-i— ... —p,J = 0. 


Hence it follows that A,=f,/. This last fact may be utilized for 
checking the computation; it is obvious that the deviation of .4, from 
a scalar matrix is a measure of the accuracy of the computations. 
Besides this final check it is convenient to utilize partial checks by 
forming the sums of the columns of the matrices B;; here the follow- 
ing relation is valid: 


I | 
@)  Sa=4Z-pnl | |s Se do-vl | |: 


] ] 
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where 5, is the column of the sums of the matrix B,;, and >» the 
analogous column of the matrix A. Finally, from the equation 


] 


l 
AB,_, =—4, =1 
Dae 


Pa 
it follows that 


(9) Aes > Bo. 


Formula (9) provides an algorithm for inverting matrices. For 
matrices of not very high order, in case it is necessary to solve the 
‘problem of finding the proper numbers and that of inverting the 
‘matrix as well, this method is very convenient. 

The number of operations necessary for obtaining the coefficients 
p; (including the computation of the matrix B,) is equal to (n—1)n3 
‘multiplications. 

In Table V is shown the scheme of computation, by D. K. Fad- 
deev’s method, of the coefficients of the characteristic polynomial 
and the elements of the inverse matrix. 

Let us pass now to the determination of the proper vectors of the 
matrix A. 

Let the proper numbers have been computed already, and more- 
over let them have proved to be distinct. Let us construct the 
matrix 


(10) Q, = AU +Ar AB +--+ +B), 


where B, are the matrices that have been computed in the process 
of finding the coefficients of the characteristic polynomial, and A, is 
the Ath proper number of the matrix A. 

It can be proved that Q, is a non-zero matrix, on the assumption 
that A,,..., A, are all distinct. 

We shall show that each column of the matrix Q, consists of the 


components of the proper vector belonging to the proper number A, 
for, indeed, 


(A,J—A)Q, = (A J—A)(AP M+ A "B, + --- +B,_,) 
= ARI + Apo} (B, —A) + Ap °(B, -—AB,) + --- +AB 
= AI—p APM —p,ap At +++ —¢ lt = 0. 


n—] 
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Hence it follows that (A,/—A)u=0, where uw is any column of the 
constructed matrix Q,, i.e., that 


(11) A, = Au , 


which equation shows that u is the proper vector. 

Observation 1. In computing the proper vectors in the manner 
described, there is of course no need to find all columns of the 
matrix Q,. One should limit oneself to the computation of one 
column; its elements are obtained in the form of a linear combina- 
tion of the columns of the matrices B; bearing the same designation, 
with the coefficients previously given. 

Observation 2. For computing the column u of the matrix Q, 
the recurrence formula: 


(12) Uy = 63 u; = Ayu;_1+5,, 


can be used conveniently; here 5; is the adopted column of the 


matrix B;, and ¢ is its counterpart column of the unit matrix. 
Then 


“= u,_}. 
As an example let us compute, for Leverrier’s matrix, the proper 
vector that belongs to the proper number A, = — 5.29870. 


TABLE VI. Determination of a Proper Vector by the Method of 
D. K. Faddeev 


2258.1433 | —2990.2530 | 1189.8295 |— 148.7675 | 308.9523 | | 
70.5604 — 48.1119 8.0822 0 30.5307 | 0.098820 


32.0618 — 14.2294 1.3785 0 19.2109 | 0.062181 
4.4283 — 1.5939 0.1751 0 3.0095 | 0.009741 


In columns I, II, III are arrayed the components of the first 
column of the matrices B;, multiplied by the corresponding powers 
of A,; in column IV are the components of the vector A3(1, 0, 0, 0). 
Column V contains the components of the vector X,; column VI, 
those of this vector after normalization. 
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§ 96. THE ESCALATOR METHOD! 


An original method for determining the proper numbers and 
proper vectors of a matrix has recently appeared under the name of 
the escalator method. This method gives an inductive construction 
by means of which, knowing the proper numbers and proper vec- 
tors of the matrix A,_, and its transpose, one can form an equation 
for the determination of the proper numbers of the matrix A,— 
obtained from A,_, by bordering—and next compute, by simple 
formulas, the components of the proper vectors for the matrix A, 
and its transpose. The application of the escalator method 1s com- 
menced by finding the proper vectors of a matrix of the second order. 
This problem is quite simply soluble. 

The great merit of the method is the presence of a powerful check, 
which makes it possible for the computor to be confident at each 
step, not only of his computations but also of the absence of a loss of 
significant figures. 

The method is based on a utilization of the properties of ortho- 
gonality of the proper vectors of a matrix and its transpose. 

The escalator method is especially convenient in case the complete 
spectrum of the matrix must be found. 

We shall not present the general induction from the sth step to 
the (k+1)th, but shall content ourselves with examining the 
transition from a third-order matrix to a fourth-order one. For 
convenience we shall designate the components of the vectors by 
different letters, contrary to established usage. We shall assume 
that all the proper numbers of the matrix Ay are real and distinct. 

Thus let A, (r=1, 2, 3) be the proper numbers of the matrices 
Ag and As, where 


(I) Ag = | a, oq dag 
431 432 433 
Furthermore, let X,=(x,, y,, Z,) and X}=(x’, y’, z’) (r=1, 2, 3) 
be the aggregate of the proper vectors of these matrices. 


2 J. Morris and J. W. Head [1]; J. Morris (2). 
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These proper vectors may be normalized so that 


X, %*2 x3 x1 oY. 2) 
4 é é 
¥1 Yo 3 Xe Yo 22 
¢ f 4 
21; 2g 3 *3 ¥Y3 23 
(2) 
v , Fd 
*) ¥, 2) X, XQ xy 
ae é a 
= 1 *2 Yo 22 y ye ys fad 
%3 ¥Y¥3 <3 21 Ze 23 


This follows from the properties of orthogonality of the proper vec- 
tors of a matrix and of its transpose, established in § 3, Paragraph 12. 
Let Ay be the matrix of the fourth order obtained from A; by 


bordering; let X= (x, y, z, u) be its proper vector which belongs to 
the proper number A. 
We have 


Ax = 441% + yo +4192 + G1 4t 
Ay = o1X+ Ggol + dogz + do4u 
Az 


(3) 


431% + Ago + 4332 + dg4t 
Au = Ag \% + ayo4 + QAggZ + Qgqtt. 


Let us multiply the first three equations of system (3) by 
x, y', Z,, respectively, and add. We obtain: 


A(xxj yy, + 2Z,) = (041%, +4914, +431Z,)% + (a1 2%) + Gooy) + a302,)¥ 
+ (449%, + dogy, + GggZ,)Z+ (4, 4%, + Goay) +.934Z,)U, 


whence, on the strength of the fact that (x’, y’, z}) is the proper vec- 
tor for the matrix A3: 


A(xx, yy, + ZZ,) = A,(xx) +yy! +227) + (a14x, + do4y, + .4942,)U 


and consequently 


(4) xxi +yy,+2z, = —7 
where 


(5) Pr = 4X, + do4y, + 434Z,- 
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Let 
(6) P, = 441%, + Agoy, + a432,- 


Then, on the strength of the orthogonality properties (2), the fol- 
lowing relation is valid: 


(7) S P,(xietyly + 2'2) = P, 

where - 

(8) P= OX t agoy + aygz = — (agg—A)u. 
Indeed, 


3 
2, (441x, +4goy, + 4432,) (xx + yy + Z,2) 
i ax) 
gy (XpX] + XoXa + XQxQ)X + ayy (XY, + X2Ya + XaYy3)Y 
#41 (% 12] +X9Z9 +%g2Z3)Z+ + + = Ay X + yoy +4432. 
Pru 

A,—A 
in accordance with (4), we obtain the following equation for the 
determination of the proper numbers of the matrix Aq: 


Replacing the expression x’x+ y’y+ z/z in equation (7) by — 


2 P,P! 
(9) a4y;—A = 2 ee i 


Equation (9) we may well call the escalator form of the character- 
istic equation, or the escalator equation. Moreover, on multiplying 
(4) by x,, y,, 2, (r=1, 2, 3) successively and adding, we obtain, by 
once again taking into consideration the properties of orthogonality 


(2): 


foo Se ee 
uo re Ayo d uo rot A,—A’ 
(10) 
, ae eS 
u = ro] A,—A 
Analogously 
UCU OP 1 5 
u’ = real A,—A’ u 7 rel A,—- i’ 
(11) 
rl ae ay 
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Thus on finding the proper number A from equation (9), we find, 
by formulas (10) and (11), the proper vectors of the matrices A, and 
A, belonging to this number, accurate to a constant factor. If we 
are to have the possibility of continuing the process, we must still 
normalize them in the sense of formula (2). 

It may be readily verified (again using properties (2)) that 


xx’ yy’ +zz' 5 FP: 


uu 1 (4, —A)? 
Consequently 
xx! + yy! + zz’ un! | ee at) aid 
gg es eels 


l 3 ppt 
a ~'+ 2 =p 


We note that if the escalator form of the characteristic polynomial] 
be denoted by /(A): 


3 PP 
(12) S(A) = -aygtAt 2 7 — 
then 
ae ee ie 
EM RE 2 aye 

Thus 

l ; 

ar aT (A) 


Without sacrificing the generality, we may consider that u= +4’, 


choosing the sign so that — + f'(A) will be positive. This gives 


lt i ————) if f’(A) > 0; 
(13) 


) — | a —) 


ipf'(a) < 0. 
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In concluding we note the check equations 


3 3 3 
> Px =a; D> Py = aynz5 > P,zp = ayy; 
J 
(14) 


3 3 3 
> Pix, = ayy; > Pry, = ang; > Piz, = 343 

r=) 3 

Aitdotdgtay = tr Aj. 


The last equations show that all the “new” elements of the matrix 
A, are used by us for the check. <A good coincidence of the check 
formulas guarantees the correctyess of the computations at each step. 
Upon conclusion of the process it is useful to verify the fulfilment of 
the orthogonality condition by the vectors that have been found. 

We have described the process only for matrices of the fourth 
order; the transition to the general casc is obvious. Given the 
satisfaction of the check equations the method guarantees very great 
accuracy, not only for all the proper numbers, but also for the com- 
ponents of the proper vectors belonging to them. 

In the case of a symmetric matrix, the escalator process is naturally 
facilitated, since all quantities marked by a prime (i.e., relating to 
the transposed matrix) will coincide with the corresponding quan- 
titics without primes. From the form of the escalator equation one 
can conclude that in this case the proper roots of the successively 
bordercd matrices separate. This circumstance greatly facilitates 
the determination of the roots, which are usually found by Newton’s 
method. 

We observe that the escalator form of the characteristic equation 
proves to be more convenient to the application of Newton’s method 
than the expanded form, since the computation of f(A) and /’(A) 
is effected very casily. 

Without going into detail, we note that in case successive escalator 
equations have identical or complex roots, the process described 
above must be somewhat modified. ! 

We shall find the proper numbers and proper vectors of the 
Leverrier matrix by the escalator method. 


1 J. Morris and J. W. Head [1]. 
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The solution will consist of three stages. 


Stage I. For the matrix Ao, 


( — 5.509882 1.870086 
0.287865 —11.811654 


the equation for the determination of the proper numbers will be 


A2 +-17.321536/ + 64.542487 = 0; 
its roots: 


A, = —11.895952, 4, = — 5.425584. 


For the check we form 4,+A. = —17,321536, and compute the 
trace of the matrix A): 


tr Ap = —17.321536. 


Following this we compute the proper vectors of the matrices A» 
and A», solving the corresponding systems and normalizing the 
vectors obtained: 


— 0.210926 | 0.210926 


0.210926 | 0.210926 | 4.679234 | 0.061767 
1.138422 | 26.638114 | 0.442009 


The first stage has been completed. 


Stage II. We form the matrix 


0.287865 —11.811654 9.711900 


( — 5.509882 1.870086 0.422908 
0.049099 4.308033 — 570) 


We copy on a separate sheet the newly introduced coefficients 
415, 4g3 and 43), @go in the form of columns, and juxtaposing them to 
the columns of the proper vectors of the matrix A,, we find the 
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quantities P; and P; (by accumulation). For convenience in 
future computations we copy them along with the proper vectors in 
scheme (I), arranging them in a row. 

We can now write the escalator equation for the matrix A3: 


24.12462] 0.503193 


F(A) = 12.970687 + 4+ —T7 sosgs9-4 + —s 4055044 ~ 


0. 


Let us determine its roots by Newton’s method, arranging the com- 
putation in accordance with the scheme: 


A —15 | —16.651}— 17.3458} — 17.3975 }|—17.397655 
— 11.895952 —A4 3.104 | 4.755 5.4498 | 5.501548) 5.501703 
— 5.425584— A 9.574 | 11.225 | 11.9202 | 11.971916| 11.972071 


12.970687+A4 |—2.029 |—3.680 |—4.3751 |—4.426813 | — 4.426968 


asst 7.772 | 5.074 | 4.4267 | 4.385061] 4.384936 
—_/xFe___ | 9.953 | 0.045 | 0.0422 0.042031] 0.042031 
— 5.425584 —1 

S(d) 5.796 | 1.439 | 0.0938 | 0.000279]—o.co0001 | 

P Ps 
(118959520? 2.504 | 1.067 | 0.8123 | 0.797059! 0.797014 
2 __| g.06 | 0.00¢ | 0.0035 | 0.003511 0.00351! 
(— 5.425584 — 4)” 

fa 3.510 | 2.071 | 1.8158 | 1.800570] 1.800525 

AA — 1.651 |—0.6948|-—0.0517 |—0.000155| 0.000000 
Thus A, = —17.397655. 


Analogously we find A, = — 7.594378 and A;= —5.300190. The 
check: A, +A. +A3 = — 30.292223. 


tr Ay = —30.292223. 


. We now pass on to the determination of the components of the 
' proper vectors of the matrices Ay and A3, which are found, accurate 
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to a constant factor, by formulas analogous to formulas (10) 
and (11). 

While so doing it is handy to set up auxiliary schemes (III) and 
(IV). 


Pry, Pix Puy 
— 1.645356 | 5.618671 |—0.191024| 4.237716 


2.068264 | 0.093231 | 0.240123{ 0.070317 
0.422908 | 5.711902 | 0.049099| 4.308033 


1 I ! 
A, + 17.397655 dy 7.594378 4, + 5.300190 


0.181762 _ 0.232473 — 0.151613 
0.083528 0.461086 ~ 7.974863 


Scheme (ITI) contains the corresponding products of the numbers 
P and the components of the vectors, and is obtained from Scheme 
(I); in the last row the check is carried out (for example 


» where for A 


2 l 
>, Pix;=4a 13); Scheme (IV) contains the factor Toi 
i=l : 
one takes the three computed roots in succession. 

The normalizing factors are now determined from Scheme (IT) 
and the two analogous schemes serving for the computation of the 
two other roots. Since /’{A;) > 0, 


D, ay, z,=2;=2D;; t=1, 2,3. 


VF'(Ai) 


On computing these, we obtain D,=0.745248, D.=0.644055, 
D3 = 0.172627. 

Utilizing the preceding schemes, we casily find the components 
of the proper vectors of the matrices As and 43 (in the final scheme 
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we copy them after multiplication by the corresponding normalizing 
factor). 


rr | te | Qe ee |S | eS | 


0.094129 |— 0.860553 |2.804268| 0.010928 |— 0.099909 [0.325572 
—0.766896 0.813572 |0.275404|—0.578409| 0.613612 0.207717 

0.745248 | 0.644055 0.172627 | 0.745248] 0.644055 (0.172627 
- 0.835026 1.114160 0.333026 0.137039| 0.182847 0.054654 P, and P* 


The second stage has been completed. 
Stage III. Having computed the quantities P; and P;, we write 
the escalator equation for the matrix Ay: 


0.114431 0.203721 
17.596207 + 4+ —Taa57655~4 + —7504378-1 
0.018201 
—5.300190—A 
and compute its roots: 
a, = “A 7.863262, ay = —17,152427, 
Ag = — 7.574044, a, = — 5.298698, 
4 
> 4, = —47.888431; tr A, = —47.888430. 


We next compute the proper vectors of the matrix A,, normalizing 
them as usual: 


x 1 Xo Xs, 


— 0.019872 0.032932 — 0.351235 1.135218 
0.169807 — 0.261310 0.328467 0.112183 
— 0.187215 0.236640 0.260927 0.070591 
0.808482 0.586694 0.045005 0.011058 
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and the proper vectors of the matrix Aj: 


x Xx; X; 


0.014058 | 0.023297 | —0.248476 | 0.803091 
0.780383 | —1.200908 1.509559 0.515564 

~ 1.140762 1.441927 1.589924 0.430123 
0.808482 0.586694 0.045005 0.011058 


The final check of the computation is the computation of the 
product of the two matrices composed respectively of the com- 
ponents of the proper vectors. Instead of the unit matrix there is 
obtained the matrix 


1.000005 —0.C00004 0.000000 0.000002 
— 0.000003 1.000004 —0.000002 —0.000003 
— 0.000002 0.000000 0.999994 0.000000 
— 0.000001 0.000000 0.000004 1.000006 


Lastly, for comparison, we state the escalator equation in the 
ordinary polynomial form: 


A4 + 47.888430A8 + 797.27877A2 + 5349.4556A + 12296.550 = 0, 


and we normalize the proper vector belonging to A, so that its first 
component is equal tounity. This gives X, (1; 0.098820; 0.062183; 
0.009741). 

We see that the coefficients of the equation here cited coincide 
with Leverrier’s data with greater accuracy than do the computa- 
tions by the other methods. The orthogonality relation is also 
satisfied with considerable accuracy. 


§27. THE METHOD OF INTERPOLATION 


The methods that we have presented in the preceding sections 
solved the problem of bringing the secular equation into polynomial 
form. The interpolation method, developed in this section, is 
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applicable to a more general case, to wit, the expansion of a deter- 
minant of the form 


(1) F(A) = 


ee ee ee oe se  @e@  @ ¢@ 


Sala) +++ San(A) 


(f,,(A) is a given polynomial in A), in particular, to the expansion of 
the characteristic determinant D(A)=|A—AJ/| and of the deter- 
minant |A— BA] where A and B are given matrices. 

The essence of the method consists in the following. Let it be 
known that F(A) is a polynomial of degree not exceeding the number 
.k. As is known from higher algebra, such a polynomial is com- 
pletely determined by its values at +1 points and may be recon- 
structed from such values by means of one or another interpolation 
formula. 

For an explicit representation of F(A), therefore, it is necessary to 
compute the value of +1 numerical determinants 


Fis(Ay) «+ SialAs) 
(2) F(A;) Poy | ee ee ee ) 1=0, Lesage 


furlAi) +++ SnalAi) 


where Ao, A,,...,A, are certain numbers chosen arbitrarily, 
generally speaking. 

The computation of the necessary determinants can be accom- 
plished, for instance, by the scheme expounded in § 7. 

For the construction of the polynomial F(A) by its values it is most 
convenient to use the Newton interpolation formula applicable for 
equally spaced abscissas A;. 

We cite the Newton formula for 4; =7,1=0,.., k: 


(3) F(A) = yO at)... itn, 


=0 
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where A‘F(!) designates the ith difference of the computed values of 
the polynomial F(A), which is determined by the recurrence formula 


AiF(t) = Ai-1F (+1) —di-lF (0). 
Let us put 


= > oni An, 


Fs m= 


A(A—1)... (A—t4+1) 
Gt — 


Then formula (3) is transformed into the form 


F(a) = > A‘F(0) (=, éniA'F(0)) am 
(4) 
k k 
= F(0)+ 5 ( = cai d'F (0) a 


This formula bears the appellation ‘‘the A. A. Markov interpolation 
formula”’. 

In Sh. E. Mikeladze’s work [1] formula (4) has been chosen as the 
interpolation formula. 

We attach a table of the coefficients c,; for m<i<20, expressed 
as decimal fractions.! 

In using the interpolation formula it is convenient as a check of 
the supporting values of determinant (1) to compute one more value 
of F(A), viz., in our case, F(k+1), for 4*+1F(0) must be equal to 
zero, and A‘F(0) and 4‘F(1) are equal to each other. 

The interpolation method requires a great many operations. 
Thus for computing the coefficients of the characteristic polynomial 
by means of interpolation formula (4), it is first of all necessary to 
compute (2+1) determinants of the nth order. This requires 
a (n—1)(n2+2+3) multiplications and divisions. If one takes 


the coefficients of the interpolation formula from a table, it is still 


(n+ 1) 


n 
necessary to carry out D 


cients of the characteristic polynomial. 


multiplications to obtain the coeffi- 


1 This table was computed by N. M. Terentiev and K. I. Grishmanovskaya. 
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The overall number of multiplication and division operations is 
thus 


n+ | (n—1)(n2-+n43) 42H) 


3 


which exceeds by far the number of operations necessary for com- 
puting the same coefficients by the method of A. M. Danilevsky or 
by that of A. N. Krylov. 

In addition, the method in question does not permit one to 
siniplify in any way the problem of finding the proper vectors of the 
matrix, whereas in computing by the methods of A. M. Danilevsky 
or A. N. Krylov the task of determining the proper vectors of the 
matrix is much facilitated. Nevertheless the interpolation method 
is interesting as a method making possible the solution of more 
general problems. 

As an example we again exhibit a computation for Leverrier’s 
matrix. 

Omitting the wearisome computation of the determinants, we 
find that 


D(0) = 12296.55, D(1) = 18492.17, D(2) = 26583.68, 
D(3) = 36894.41, D(4) = 49771.69. 


We next compose a table of differences: 


D(A) 
12296.55 
6195.62 


18492.17 1895.89 
8091.51 


26583.68 2219.82 
10310.73 
36894.41 2566.55 


12877.28 
49771.69 


(We note that in case one is computing the coefficients of a char- 
acteristic polynomial, the computation of superfluous values of 
D(A) for a check need not be done, since in this case a reliable check 
is the equality 44F(0) =(—1)n!). 


OBI ON & COND oe TOA 


10 


/3 


TaBLeE VII. 
] 2 
1.00000 00000 
—0.50000 00000 0.50000 On0n0 
0.33333 33333 —0.50000 00000 
—0.25000 00000 0.45833 33333 
0.20000 00000 —0.41666 66667 
—0.16666 66667 0.38055 55556 
0.14285 71429 —0.35000 00000 
—0.12500 00 0.32410 71429 
O.LL111 LILIL = =—0.30198 41270 
—0.10000 00000 0.28289 68254 


0.090909 09091 
— 0.083333 33333 
0.076923 07692 
— 0.071428 57143 
0.066666 66667 
—0.062500 00000 
0.058823 52941 
— 0.055555 55556 
0.052631 57895 
— 0.050000 00000 


—0.26626 98413 
0.25165 64454 
—0.23870 85137 
0.22715 24111 
—0.21677 08218 
0.20738 93121 
—0.19886 64114 
0.19108 62513 
— 0.18395 30567 
0.17738 69829 


3 


0.16666 66667 
— 0.25000 00000 
0.29166 66667 
—0.31250 00000 
0.32222 22222 
—0.32569 44444 
0.32551 80776 
— 0.32316 46825 
0.31950 39683 
— 0.31506 77910 
0.31018 99952 
— 0.30508 41751 
0.29988 87242 
— 0.29469 38553 
0.28955 82939 
— 0.28451 98560 
0.27960 22978 
—0.27481 98358 


4 


0.041666 66667 
— 0.083333 33333 
0.11805 5 
— 0.14583 
0.16788 
—0.18541 
0.19942 
— 0.21067 
0.21974 
—0.22707 
0.23301 
— 0.23781 
0.24169 
— 0.24481 
0.24730 
—0.24925 
0.25077 


99694 
70858 


5 


0.83333 33333- 
56 —0.20833 33333- 
0.34722 22222- 
—0.48611 1)11I- 
0.61863 42593. 
—0.74218 75000- 
0.85601 30071 - 
—0.96024 16777. 


0.10554 11339 
—0.11422 22865 
0.12214 17270 
— 0.12937 15303 
0.13597 89675 
—0.14202 54343 
0.14756 6258) 
~—0.15265 09436 


Table of Coefficients of A. A. Markov’s Interpolation Formula 


10-2 
10-1 
10-2 
10-1 
10-1 
10-1 
10-3 
10-3 


0.13888 88889. 
— 0.41666 66667- 
0.79861 1L111- 
—0.12500 00000- 
0.17436 34259. 
0.22598 37963. 
0.27848 62305- 
— 0.33092 89572. 
0.38267 76988. 
— 0.43331 40432- 
0.48257 04949. 
— 0.53028 48954. 
0.57636 84942. 
— 0.62078 35389. 
0.66352 74910. 


10-2 
10-2 
10-2 
10-1 
10-1 
10-3 
10-1 
10-2 
10-3 
10-3 
10-3 
10-1 
10-2 
10-1 
10-1 


3 


/ 
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7 


0.19841 26984. 
— 0.69444 44444. 
0.15046 29630. 
— 0.26041 66667- 
0.39525 46296- 
—0.55063 65741- 
0.72250 00919- 
— 0.90727 07690- 
0.11019 04517- 
—0.13038 56762- 
0.15110 24302. 
—0,17216 81227. 
0.19344 18265. 
—0.21480 89121- 
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8 


0.24801 58730- 
— 0.99206 34921. 
0.23974 86772. 
— 0.45469 57672. 
0.74618 33113- 
—0.11123 51190. 
0.15489 69028- 
— 0.20505 51606- 
0.26110 82453- 
~— 0.32244 63934- 
0.38847 84994- 


— 0.45864 70640 
0.53243 56241 


10-4 
10-4 
10-3 
10-3 
10-3 
10-2 
10-2 
10-2 
10-2 
10-2 
10-2 
-10-2 
-10-2 


9 


0.27557 31922.10-5 


— 0.12400 79365. 
0.33068 78307. 
— 0.68204 36508. 
0.12035 65917- 
—0.19121 33488. 
-1Q-3 
—0.39228 17198- 
0.52279 94100- 
~0.67289 18835. 
0.84193 88893. 
—0.10291 65477- 


0.28173 03941 


10-4 
10-4 
10-4 
10-3 
10-3 


10-3 
10-3 
10-2 
10-3 
10-2 


10 


0.27557 31922. 
-10-5 


—0.13778 65961 


0.40187 75720. 
— 0.89561 28748- 
0.16913 30467. 
-10-4 


—0.28533 30761 


0.44358 12552- 
— 0.64824 21930- 
0.90267 28545. 
—0.12093 17380- 
0.15698 20955- 


10-6 


10-5 
10-5 
10-4 


10-4 
10-4 
10-4 
10-3 
10-3 


0.25052 10839. 
—0.13778 65961 - 
0.43632 42210- 
—0.10448 81687- 
0.21027 76553- 
— 0.37546 84744- 
0.61431 22437. 
— 0.94031 83373- 
0.13659 18875. 
—0.19022 81621- 


10-7 
10-6 
10-6 
10-5 
10-5 
10-5 
10-5 
10-5 
10-4 
10-4 


fa 
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0.20876 75699. 
— 0.12526 05419. 
0.42797 35183. 
— 0.10960 29742. 
0.23417 63229. 
— 0.44126 50535- 
0.75803 49082. 
—0.12130 42722. 
0.18353 50023- 


10-8 
10-7 
10-7 
10-8 
10-6 
10-6 
10-6 
10-5 
10-5 
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0.16059 04384- 
— 0.10438 37849- 
0.38274 05448. 
— 0.10438 37849. 
0.23599 43405- 
— 0.46803 07957- 
0.84236 33371- 
— 0.14067 66531- 


10-9 
10-8 
10-8 
10-7 
10-7 
10-7 
10-7 
10-6 


14 


0.11470 74560- 10-1° 
— 0.80295 21918-10-1° 
0.31448 96085. 10-8 
—0.91001 24841-10-° 
0.21705 35904- 10-8 
—0.45196 17150- 10-8 
0.85054 52978-10-§ 


15 


0.76471 63732-10 3# 
—0.57353 72799- 10-12 
0.23897 38666 - 10-10 
— 0.73126 00319. 10-10 
0.18351 59980-10-® 
—0.40032 10556-10-° 


0.47794 77332-10-15 
— 0.38235 81866- 10-12 
0.16887 48657-10-11 
— 0.54486 04159- 10-21 
0.14351 97385- 10-26 
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17 


0.28114 57254- 10-34 
—0.23897 38666 - 10-25 
0.11152 11378-10-1% 
— 0.37837 52888-1032 


18 


0.15619 20697 - 10-35 
—0.14057 28627- 10-14 
0.69114 99084. 10-14 


19 20 


0.82206 35247- 10-27 
— 0.78096 03484. 10-16 0.41103 17623. 10-18 
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Lastly we compute the coefficients of the characteristic poly- 
nomial, arranging the computation in accordance with the scheme: 


A'D(0) Cat 


6195.62 ].00009000 
1895.89 0.50000000 |—0.50000000 
323.33 0.16666667 |—0.50000000 | 0,33333333 
24.00 } 0.04166667 |—0.25000000 | 0.45833333 |—0.25000000 
47.8883 797.280 5349.45 


The coefficient p, = D(0) = 12296.55. 

The interpolation method is always applicable; in particular the 
case in which the characteristic polynomial has multiple roots is not 
a whit distinguished from the other cases. 

If instead of the numbers 0, . . . , A the numbers A; =a + At be taken 
as the interpolation nodes, formula (4) will thereupon be changed 
in form as follows: 


(5) F(A) = F(a)+ > (> 6, ht AiF (2)) (A—a)™, 


It may sometimes be convenient to take as interpolation abscissas 
numbers that are not equally spaced. In this case one can use the 
general interpolation formula of Newton’s. However in case of 
abscissas not equally spaced it is more convenient to construct the 
required polynomial by the method of undetermined coefficients, 
ViZ.:= 

F(A) = ag +a,At-1 + -- + +a, 


Then for the determination of the numbers a,,7=0,..., &, we will 
obtain a system of algebraic equations 


F(A,) = aod tayAp t+ +++ +a, 


which may be solved by any of the methods set forth previously. 
The interpolation method may be employed conveniently, in 
particular, to the expansion of the determinant |4 — BA| in case the 
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matrix B has a small determinant. If, however, the determinant 
of B is not a small number, the coefficients of the desired polynomial 
can better be determined by means of the transformation 


|A—BA| = [B| |AB-1—Al |. 
The matrix AB-! may be found by the climination method (§ 12). 


§ 28. COMPARISON OF THE METHODS 


In the foregoing sections there have been expounded six dif- 
ferent methods for bringing the characteristic determinant of the 
matrix into polynomial form; the determination of the proper 
numbers of a matrix is thus reduced to the determination of the 
roots of an algebraic equation. Four of these methods also make 
possible the determination of the proper vectors belonging to the 
proper numbers of the matrix, avoiding the solution of the linear 
systems defining the components of these vectors. 

We shall endeavor to characterize the peculiarities of these 
methods. 

The least number of operations for the entire process of computa- 
tion is required by the method of A. M. Danilevsky. ‘The Danilev- 
sky scheme secures a comparatively high accuracy for the sought-for 
coefficients, which are all determined simultaneously and therefore 
with approximately the same accuracy. The coincidence of the 
coefficient p,; with the trace of the matrix serves as a good final check 
of the accuracy. The method makes possible the determination of 
the proper vectors easily enough, too. The somewhat complicated 
“pattern” of the computational scheme is easily mastered by the 
computor. Lastly, the presence of multiple roots of the character- 
istic equation is no obstacle to the use of the method. 

The method of A. N. Krylov requires a somewhat greater volume 
of computations than does that of A. M. Danilevsky. This method 
can nonetheless be recommended for wide application thanks to the 
simplicity and compactness of the computational scheme. 

There is a certain shortcoming of the method: in many cases the 
linear system for the determination of the unknown coefficients 
proves to be inconvenient owing to the cocfhcients of this system 
being of different orders of magnitude. This may lower the accuracy 
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of the computation. Moreover in case there are multiple roots of 
the characteristic equation, the method permits the determination 
of the coefficients of the minimum polynomial only. 

The Samuelson method differs little in its idea from the A. N. 
Krylov method. Its computational scheme may sometimes be 
useful thanks to the complete uniformity of operations. 

The escalator method solves a somewhat more general problem 
than do any of the rest of the methods described, viz.: we find not 
only all the proper numbers of the matrix, but also all the proper 
vectors of the matrix itself and of its transpose as well, by the very 
nature of the process. The overall number of operations necessary 
for the solution of this problem cannot be reckoned, since in the 
course of the process one has to find the roots of an (n—1)th 
algebraic equation of degree from two to n. And the solution of 
the algebraic equations—which is best carried out by Newton’s 
method—can require a different number of operations in different 
cases. The escalator method is, unconditionally, substantially more 
labor-consuming than are the three methods considered above, but 
it makes it possible to obtain significant accuracy and guarantees 
the reliability of the result. 

The method of Leverrier, and the interpolation method, can 
hardly compete with the first three methods described, for the num- 
ber of operations required for these two methods is substantially 
greater. The value of the interpolation method consists in its 
generality. The method of Leverrier in the modification presented 
here is convenient for matrices of low orders. 


§ 29. DETERMINATION OF THE FIRST PROPER 
NUMBER OF A MATRIX. FIRST CASE 


In connection with problems of mathematical physics it is often 
necessary to find only a few of the first proper numbers of greatest 
modulus, and only the first proper number need be found to a 
significant degree of accuracy. 

To the solution of this problem it is convenient to apply iterative 
methods that in respect of technique recall the iterative methods for 
the solution of a system of linear equations. In using iterative 
methods the dominant proper number is obtained in the form of a 
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limit of some sequence that is constructed by a uniform process. 
The iterative method also gives one the possibility of computing the 
proper vectors belonging to the proper numbers of largest modulus. 

In expounding the iterative methods we shall not consider the 
question in all its generality, but shall limit ourselves to a discussion 
of certain cases that are particularly important in applications. 

In this section we shall assume that the dominant proper number 
of the matrix is real, and that the elementary divisors connected 
with it are linear. For simplicity of exposition we shall assume in 
addition that all the rest of the elementary divisors of the matrix are 
linear, although the derivations that we shall make would hold even 
without this assumption. 

The case we consider is that met most frequently in practical 
problems. In particular, symmetric matrices are subsumed under 
this case, as was established in § 4, as are also matrices all of whose 
proper numbers are distinct. 

On the strength of the linearity of the elementary divisors, to each 
proper number of the matrix A will correspond as many linearly 
independent vectors as its multiplicity. Let 4,, Ap, ..., A, be the 
proper numbers of the matrix A, arranged in order of diminishing 
modulus (some of them perhaps equal); let X,, Xo,..., X, be the 
proper vectors corresponding to them. If all the numbers A; are 
distinct, the vectors X; are determinate but for a constant factor; if 
two of the proper numbers are equal, any linear combination of the 
proper vectors belonging to these numbers will also be a proper 
vector. It is therefore possible so to choose the proper vectors of a 
matrix and its transpose that the orthogonality and normality 
relations (in the sense of § 3, Paragraph 12) are satisfied. 

An arbitrary vector Yo may in this case be represented uniquely 
in the form 


(1) Yo = a,X,+a.Xo+ +--+: +4,X,, 


where the numbers a; are constants, some perhaps zero. 
Form the sequence of vectors 


(2) AY,, A2¥o,..., At¥o, .. « 


We shall call the vector A‘Y, the Ath iterate of the vector Yo by the 
mainx A, 
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Obviously 
(3) AY = QA, X 1 + GohoXot+ eee +a,A, Xx 
(4) AbYy = a4\X,+aghgXqt +++ +4,A4X,. 


We use the notation At¥Yo=Y,=(yig, you, ~~ + >¥Yes), and shall 
elucidate the structure of the components of Y,. Let 


x) = (¥3)) KOlye rey Xar)s 


No = (X12) Xe0,-- + Xq2)s 


x, = (X19 Monsees Xan): 
We then obtain from (4) 


(3) Yu = ax; 4} + 03x;0A9-+ eee +4,X;Ah5 


where the coefficient of A‘ reduces to zero when, and only when, 
a,;=0Oorx;;=0. The coefficients of Ai in all the components of the 
vector Y, can equal zero only if a,=0, since the vector X, is not 
zero. 

Thus any component of the vector Y, depends linearly on 


oe A‘, Let us denote any of these components by y, (omitting 
the first index). 

Then 
(6) Ye = CA teedat «++ +6,A4, 


where the coefficients ¢; do not depend on the index k. 

If among the numbers A; some are equal, the corresponding terms 
may be combined. Thus if A, is a proper number of multiplicity r, 
then 


(7) yp = Ai +64 att oe. teat 


We shall now assume that the vector Yg has been so chosen that 
a,#0!. Then 2} appears with coefficients c,#0 in at least one 
component of the sth iterate of the vector Yo. 

1 Since a;=(X{, Yo), where X; is the first proper vector of the transposed matrix, 


the requirement that a,#0 will be fulfilled in case the vector Yo is not orthogonal 
to the vector .X7. 
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Let us consider three cases: 
I) Jaa] > [Ae 
) A adeeris She A Saad 
3) ay = —de; asl > |Aal- 


In the first case we have 


Yarr — Cay +egAgt + +++ +e,an 


UT cat teode t+ eee +c,As 
(8) 
9 | + b0g* 1 + byag” + +++ +b") 
OT Da bye t byog toes tbo, 
where 
(9) a ee 
C1 I 


Carrying out the division and retaining the terms up to the order 
of a2‘ and aj inclusive, we obtain 


(10) a = A,[] diet — boat + 5,6502"] + O(at +02), 
where 
(11) bo = bo(1—ae), 63 = 63(1 —a3). 


Hence we sec that if & is sufficiently large, 


(12) Ay = 24), 
4; 

1.¢., the first proper number is approximately equal to the ratio of 
any corresponding components of two neighboring and sufficiently 
high iterates of an arbitrary vector by the matrix A. 

In carrying out the iterations in practice, one should compute 
the ratio Jatt for several components. <A good coincidence of these 

rt 

ratios will show that in expression (10) the difference of values of 
the coefficients 5, bg has already ceased to play a significant role. 
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The rapidity of the convergence of the indicated iterative process 
is determined by the magnitude of the ratio ze and may be very 
slow. 

Sometimes in computing the iterates it is expedient to divide at 
each step the components of the vectors being iterated by the first or 
largest component, or to normalize them in the customary manner, 
to avoid growth of the components. In doing this, rather than 
the sequence Y, we obtain a sequence Y,=y,A*¥o, where y, is the 
normalizing factor, and to obtain A; one must take the ratio of the 
components of the vectors AY, and FY,. 

It may occur, although this is improbable, that the initial vector 
Y, has been unfortunately chosen, viz.: such that the coefficient a, 
is equal to zero, or is very close to zero. In this case, at the first 
steps of the iteration the preponderant term will be that dependent 
on Az (if ag#0). However even if a, is equal to zero exactly, after 
several steps of the iteration the saummand dependent on 4,, thanks 
to rounding errors, will appear at first with a very small coefficient; 
in proportion as the iterations proceed, this summand will grow 
with satisfactory rapidity by comparison with the rest. A “struggle 
for dominance” of the terms depending on A, and A, much darkens 
the picture and makes necessary the replacement of the initial vector. 

Example 1. Let us try to determine the first proper number of 
the Leverrier matrix. 

We will take the vector (1, 0, 0, 0) for the initial one and form 
20 iterates, normalizing them at each step. 

We subjoin only the last two iterates: 


F 45 AP i» Ratio of the 
components 
1.00000 — 17.4655 — 17.466 
— 8.20321 143.3809 — 17.479 
8.17013 — 143.0881 — 17.514 
— 7.95957 149.2676 — 18.753 
— 6.99265 132.0949 


From the table we see that the ratios of the different components 
are still far from each other; this shows that the process has not yet 
stabilized. Indeed, accurate to three decimals, 4, = — 17.863. The 
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process of iteration converges slowly because of the fact that the 
second proper number A.= — 17.152 differs little from the first in 
modulus. 

Example 2. Let us determine the first proper number of the 
matrix 


0.287865 —11.8)1654 3.711900 


— 5.509882 1.870086 0.422908 
( 0.049099 4.308033 —12.970687 


We shall take (1, 0, 0) as the initial vector. We display a table 
of the iterates commencing with the 12th: 


Pie AP, Pig APs Pig AP,, 
1.0000000 —17.351783 1.0000000 —17.378482 1.0000000 —17.389552 
—8.)139091 141.126754 —8.1332710 141.483894 —8.1413264 141.632991 
7.8783245 — 137.093170 7.9008117 — 137.468256 7.9102568 — 137.625469 


ce ae mr 


0.7644154 —13.318201 0.7675407 —13.362844 0.7689304 — 13.382030 


We find the ratios of the corresponding components for the 12th, 
13th, 14th and 15th iterates to be 


—17.351783 —17.378482 —17.389552 
—17.393189 —17.395694 —17.396795 
—17.401310 —17.399257 —17.398356. 


The three last ratios enable us to consider A, to be —17.39 or 
—17.40. As we have seen (§ 26), accurate to four figures, 
A, = —17.3977. 

In case the largest proper number is multiple, we have from (7), 
analogous to the expression we had before, 


Yet _ 3 o(4:#). 
¥; a Ay 


Thus in this case too, under the condition that a,40, the ratio 


gives the approximate value of the largest proper number. 
k 

The question of the multiplicity of the root cannot be solved without 
a more detailed investigation. We shall return to this question 


again in § 31. 
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Example 3. Let us determine the first proper number of the 
matrix 


1.022551 0.116069 —0.287028 —0.429969 
0.228401 0.742521 —0.176368  —0.283720 
0.326141 0.097221 0.197209 —0.216487 
0.433864 0.148965 —0.193686 0.006472 


On solving the characteristic equation 
A4 — 1.968753A3 + 1.391 18472 —0.4152914+.0.044360 = 0, 
we obtain for the proper numbers the values 
A, = Ao = 0.667483, As = 0.346148, A, = 0.287639. 
Let us determine A, by means of the iterative process, taking for 


the initial vector (I, 1, 1, 1). 
We give a table of the iterates beginning with the 9th: 


Po AY, Pio AP io Pi AY, 
1.000000 0.666160 1.000000 0.666822 1.000000 0.667151 
1.844723 1.230507 1.847165 1.232545 1.848387 1.233563 
0.676506 0.449420 0.674643 0.44921! 0.673660 0.449088 
0.875250 0.583298 0.875613 0.584025 0.875834 0.584399 
4.396479 2.929385 4.397421 2.932603 4.397881 2.934201 


The ratios of the components of these iterates are computed to be 


0.667151 
0.667373 
0.666639 
0.667249. 


0.666160 
0.667042 
0.664325 
0.666466 


0.666822 
0.667263 
0.665850 
0.666990 


The last four ratios give for A, the value 0.667, which is correct to 
three places. 

Let us analyse, lastly, the third case, that for which A, = —do, 
[41] > [As]. 
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From equation (6) we sce that in this case the even and the odd 
iterates have different coefficients of corresponding powers of 4, 
since 


ak 


(ey +co)Ap + cgdi + --- +6,22 


Yox 


Youre = (¢4—¢2) Ap’? +3437 1 + ae eae, 


and two neighboring iterates thus cannot be used for the determina- 
tion of A,. Wecan, however, determine 4} by one of the following 
formulas: 


9 + ” DE+ 
ew PEE oor Bw BEN. 
2k You-1 


In all the cases we have examined, for the iterative process to be 
successful it is necessary that the first proper number sufficiently 
exceed the one next following, in point of modulus. 

The process that we have described also makes possible the deter- 
mination of all the components of the proper vector belonging to 
the largest proper number, for the ratios of the components of the 
vector Y, tend to the ratios of the components of this proper vector. 

Indeed, for 2,0: 


Y, = ALY 5 


ilaX, +ao(32) X24 eis +04) Xo] 
Ay Ay 


k 
aah [x + (3) x, | 


If A, be a proper number of multiplicity r, the indicated method 
makes it possible to determine one of the proper vectors belonging 
to A;. We note that in this case, by proceeding from different 
initial vectors, we arrive, generally speaking, at different proper 
vectors. 

In the third case, when A, = — Aj, it is expedient to construct the 
vectors ¥,+4,Y,_, and Y,—4,Y,_,;. The ratios of the components 
of these vectors will respectively tend to the ratios of the components 
of the vectors X, and X, belonging to the proper numbers A, and Ag. 
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Indeed, in view of the equation 


Y,= a, A,X, +42(—A,)*X_+agAgX 3+ eas 
we have 


Y,+A,Y 4-3 = 9a, AiX, +a3(Agt+A,)Ag 1X3+ oes 


= Ai(20,%, + (3) ) ; 
1 


Y,-A\Y,.1 = Qa2(— Ay)tX2+ag(Ag—A,)Ag 1X3 + eee 
k 
a (—Ai)4( 2002 +.0() ) 
1 


As an example let us find the proper vector belonging to the first 
proper number of the matrix of Example 2. In § 26 we determined 
the components of this vector, viz.: 


X, = (0.094129, —0.766896, 0.745248) 
or, after normalizing, 
X, = (1.00000, —8.14729, 7.91730). 


From the table exhibited in Example 2, we find for the com- 
ponents of the vector the following values: 
1.00000 1.00000 1.00000 
—8.13327 —8.14133  —8.14472 
7.90081 7.91026 7.91426 


We see that the last result already approximates closely enough 
the exact value. 


In concluding this section we shall find the first proper number, 
and the proper vector belonging to it, of the matrix 


0.22 0.02 0.12 0.14 
0.02 0.14 0.04 —0.06 
0.12 0.04 0.28 0.08 
0.14 —0.06 0.08 0.26 
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In Table XIII, § 17, 14 iterates were computed, starting with the 
vector (0.76; 0.08; 1.12; 0.68). 

Computing the ratios of the components of the 14th and 13th 
iterates, we find for A, the value 0.4800. 

(We ignore the second component of the iterates because of its 
smallness by comparison with the other components.) 

The ratios of the components of the 7th and 6th iterates give for 
A, the values 


0.4800 0.4792 0.4808. 


For the components of the proper vector we find from the 14th 
iterate the following values: 


(1.0000 0.0000 1.0000 1.0000). 


It is readily verified that the exact value of A, = 0.48, and that the 
proper vector belonging to it has the components (1, 0, !, I). 


§ 30. IMPROVING THE CONVERGENCE OF 
THE ITERATIVE PROCESS 


The convergence of the iterative process may be very slow. 
The cause of this is sometimes the presence of a nonlinear elementary 
divisor corresponding to the first proper number. In § 3¢ it will be 
shown how to reveal this circumstance and how to proceed in such 
acase. But the cause of the slow convergence may also be the close- 
ness of the second proper number to the first. In this case the con- 
vergence may be improved by means of certain devices. 

1. Ravsing the matrix toa power. It is sometimes expedient to apply 
Lobachevsky’s method to the characteristic equation of the matrix. 
This can be done without finding the characteristic equation in 
explicit form, in the following manner. Successive powers of the 
matrix A: A2, At, A8, A'6,... are computed. For each of these 
matrices the trace is computed, and then A, is determined approxi- 
mately by the formula 


(1) A, = V/tr A®. 
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The last formula follows from the fact that 
trA”™ = AT + A+ +>: +AR, 


and consequently 


— 9 (Alt... a fe" 
VYuwa = a fis (2) +(3) 


- sof’ 


Instead of extracting the root one can begin the usual iteration of 
an arbitrary vector, using the constructed powers, so that knowing, 
for instance A’, A8Y, can be easily constructed, and then A®(A8 Yq) 
=Al6y, and finally, A!7¥9=A(A!*®Yo), and A, then found as the 
ratio of the components of the vectors 417Y,) and A!6Y5. 

Obtaining each power of a matrix requires, however, 23 opera- 
tions, and is therefore equivalent to n iterations of the vector. Ifthe 
convergence of the process of iterations is good, sufficient accuracy 
in the determination of A, will be attained after a few steps and there 
will be no need to have recourse to the process described. One 
should turn to it, however, if the number of iterations for attaining 
the requisite accuracy exceeds n- logon. 

In computing the powers of matrices one should use the check 
described in § 25. 

2. The scalar product. This device is particularly handy in 
application to a symmetric matrix; we will expound it here without 
that assumption, however. 

Together with the sequence of iterates of the vector VY, by the 
matrix A, 


(2) 


(3) Yo, Y; = AYo, Yo = A®Vo,..., Vy = A*Vo,... 


let there also be computed the sequence of iterates by the matrix 4’, 
the transpose of A, 


(4) Yo, ¥i = A’Yo, Va = A’2Y,..., Vi = A'S, 


Let 5,,..., 4, be the coordinates of the vector Yo with respect to 
the basis X},..., X,; and a,,..., 4, the coordinates of Yo with 
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respect to the basis X,,..., X,- We assume in addition that the 
bases are so selected that the system of vectors X,, Xe,..., X, and 
X},-.+,X, satisfies the conditions of orthogonality and normality 
in the sense of § 3 Paragraph 12. 

Let us form the scalar product (Yj, Y,): 


(Yi, Vz) = (AY, Ab¥o) = (Yo, 47 Vo) 
= (bX, +b.Xo+ ++ +8,X3, 
a, AP X, + agA'X, + oes +a,A24X ), 


A further step is made possible by the properties of orthogonality and 
normality of the system of vectors X,,..., X, and X;..., X4: 


(5) (Y4, ¥,) = 016,02 + agbod3 + --- +4,b,4%. 
Analogously, 
(6) (Vien, Vg) = aby AP7 4 gb A + ©. 40,6 AO, 


From equations (5) and (6) we obtain 


(Yi, Ys) __ a, b,Ar* + abode’ + +++ +anb,Am 
(Yi-a Ya) a,b,ae) + agboAtt 4... +a,bas-! 
(7) 
Ao\ 2 
= 4140(7) 


From this estimate it is evident that the formation of the scalar 
product lessens the number of steps of the iteration required for the 
determination of A, to a given accuracy by almost half. However 
along with this, sequence (4) must be computed in addition. 

In case of a symmetric matrix, sequences (3) and (4) coincide, and 
thus the application of the scalar product method is particularly 
efficient. Beginning with a certain step of the iterations, one must 
compute the corresponding scalar products and determine 4, by 
their ratio, to wit: 


A, ~ (A‘Yo, AtYo) 
1 (At-1Yo, AEYg) 
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Example. Let us consider the matrix 


1.0000000 0 1.0000000 0 

1.0000000 0.7777778 0.3333333 0.3333333 

W) — 0.0252525 0.5555556 —0.0252525 |» 
0 —0.8888889 —8.6444444 O.V11111) 


whose proper numbers are, |, 3, 4 and 4. 
For the determination of A, we form the iterates A‘Yo, taking as 
Y, the vector (1, 1, 1, 1). 

Al7Y, A\8y, Alsy, A2y, 
4.6731097 4.6760089 4.6779433 4.6792336 
8.3733415 8.3912886 8.4032694 8.4112637 
0.0028992 0.0019344 0.0012803 0.0008605 

—8.3861607 —8.3998278 —8.4089592 —8.4150555 


4.6631897 4.6694041 4.6735438 4.6763023 


The last row maintains a check of the computations. 
The ratios of the three corresponding components of these 


iterates will be 
1.000414 1.000276 


1.001428 1.000951 
1.001087 1.000725. 


The last column gives 4,;~ 1.001; the value found coincides with 
the exact one accurately to one unit in the third place. 
We shall now show how this value can be refined by an applica- 
tion of the scalar product method. 
With this in view we form iterates of the vector (1, J, 1, 1) by the 
transposed matrix A’. 
On computing them, we obtain 
A’20Y, = (0.7961118, —0.0002189, 3.9939022, —0.1134904). 
Furthermore 
(A20Y,, A’20Y,) = 4.681817 and (A!®¥po, A’2°Y,) = 4.681816. 
The ratio 
(A°°¥o, A’20Yo) 
(A)°¥o, A’2°Y 5) 


gives A, the value 1.000000, correct to six decimals. 


= 1.000000 


Improving Convergence of Iterative Process 215 
Observation. If in finding the iterates we normalize them, then 


_ (AY, 1, AY) 


A, = , 
"(Ppa AY) 


where normalization of vectors A’*Yq is a matter of indifference. 

3. The 62 process.1 This device is applicable only in case 
JA,| >A], and A,, Ag and Ag are real. 

Let us assume that we have determined a number of quantities 
(8) Yas Yaris Yetar- + +s 


about which it is known that 
(9) y= Ak teodgt aad +e i. 


As y, one can take, for instance, any component of the vector 
Y,=AtYo, the trace of the matrix A‘, the scalar product of corres- 
ponding iterates, etc. Then, as has been shown in § 29 and § 30, 
the first proper number A, can be approximately determined as the 
ratio 4, = Yi+1, 

k 


Moreover it was shown in § 29 that 
(10) u, = A,[1 — byoeh — byes + 56,6502] + O(at + 02+), 
where 


bs 


Bey. bi i sia 
C1 (1 Q»), b, = os (1 a3) and Qg= is 


If the convergence of the sequence 4, 444), Uj+2,..- is insuffi- 
ciently rapid, it can be greatly improved by the following device, 
which is called “‘the 62 process”’. 


Let us form 
ad Up+] 
(11) P(u,) = Yeti Servet 


ty — QUjpty + ty+e 
LA. Aitken [1]. 
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We shall show that 


(12) P(y) = 4,+0(7) "+ o(}:)" 


With this aim in view we put 


u, = Ay(L+&), 
then 
1 + Ey ] + Ex+1 
P+ e+. 1+ &+0 


uy Ug+y 
Ug+y gto 


— 22 


1 


Separating the last determinant into the sum of four, we obtain 


uy ed é Ept 
= AM le —2eeit etet | * - 
Ut. Upto Ecti +e 
But 
Uy — Qtyay titre = Ayleg—2egta + ere}. 
Thus 


€y Ext 
Egta Egtey | 
&, _ 2 Ep +1 + Egt+e 


P(u,) = r) + 


On calculating we find that 


Ey Ext) 
Egt+1 Ege 
Ep —2Egti t Epte 


~ Aa’ + Ba2*, 


where 
— b. —_— Of 2 7.2 
A= ae B = 646403. 
Thus 
k 2k 
P(u,) = 4,+0(3) +0(7) 
1 1 


Hence it follows that the error in the determination of 4, with the 
aid of the 62 process may be less by far than in a direct determination 
of it from the sequence 1%, u,44,... 

Observation. In finding the first proper number of a symmetric 
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matrix, the 62 process must be applied not to the components of the 
iterates, but to corresponding scalar products. 

Lastly we note a property of the operation P(u,) that is convenient 
for practical computations, viz.: 


(13) P(upte) = ¢+P(u,). 


Hence it follows that before employing the 6? process, one can deduct 
a convenient constant from the numbers 44, 444), U+2. (Such a 
constant could be, for example, one consisting of the number of 
decimals that have already been established.) 

We shall show the application of the 6° process to the examples of 
§ 29. 

In Example 2, § 29, for the ratios displayed the application of the 
6° process gives for A, the following values: 


— 17.3974 
— 17.3977 
~ 17.3977. 


Accurate to four decimals, A, = — 17.3977. 
Analogously, in Example 3, § 29, the application of the 6° process 
to the displayed ratios gives for 4, the following values: 


0.66748 
0.66748 
0.66748 
0.66748. 


Thus 4, is already determined with an accuracy of five decimals (the 
exact value of 4, =0.667483). 

The 6? process may also be applied to the determination of the 
components of the first proper vector. In this connection we shall 
show two distinct variants of this process, the choice depending on 
whether A, can be considered as known to a sufficient degree of 
accuracy or not. 

|) Let there be known to us only the sequence of iterates A‘ Yo, 
where for computational convenience let each iterate be normalized 
by division by a fixed component z,. 

We shall denote any other component of the vector Y, by yx. 
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If 


Y_ = CAL cgAbt+ +++ +eqdt 


2, = dA + boAgt +++ +5,A8, 
then the indicated division reduces z, to unity and y, to »,, where 


_ Me _ CAL Heed t +++ +eqae 
Zp DAL + bgAgt +++ +beAn 

sige Gabi ats 52 (3) Fak 

St ee 

A simple computation shows that 

= 1 40(43)' (3)". 
Pa) = by +0(7) si A 

Thus if the 6® process is applied to all components of the normal- 
ized vector A‘ Yo, we shall find the ratios of the coefficients standing 
with the powers of A, in the expressions for y,. These coefficients, 
moreover, are proportional to the components of the proper vector. 


Thus for the proper vector of Example 2 of the preceding section, 
application of the 62 process gives for the components of the vector 


the values 
1.00000 
( - si , 
7.9172] 
which are considerably closer to the exact ones than the values com- 
puted directly from the same iterates. 

2) If A; is known with sufficient exactitude, an improving process 
can be constructed in the following manner. We multiply all the 
components of the vectors A&-1Yp, AtYo, Aé+1¥o, by ay, 1, Ay? 
respectively, and then apply the d® process to them. Since 


nara, = Mey +As lAyco+ oes + At Aye, 


a Mie + Agee + vee Ake, 


ato tte a 
Yet1A) = Ae, + —=— tg+ cee bf = ¢,, 
Ai A, 
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we have 
Aiy- y asic 
ms eye = (creed ‘Ne "(Ay — An)? + ae 
k 
eveady Media)" +0(F) | 
1 
Moreover 


k-1 k-1 
Aye-1—2ypt 2 "Yer =08 — (Ay— Ag)? + +++ +e, (A, —4,)?. 


Hence 
| AiYa-1 Vy 


Yp Ay yess k | ( ) 
= ¢,4,11+O0[—]}] |- 
Ye-1 — 24 +941 Ay 


The ratios of the numbers obtained, which latter are computed for 
the different components, give the ratios of the components of the 
proper vector. 


§ 31. FINDING THE PROPER NUMBERS 
NEXT IN LINE 


In this section we shall show that by modifying the process of 
iteration in a certain manner, one can determine the product 
of several proper numbers. This opens up for us the possibility of 
determining one after another the proper numbers next in order 
after the first. It should be remarked, however, that the deter- 
mination of the product of even two of the proper numbers en- 
counters great difficulties, chief of which is the disappearance of 
significant figures. As a rule, therefore, even the second proper 
number determined by means of the iterative process is of a much 


lesser degrce of accuracy than the first. 


Let 
(1) Y_ = CAL HeodAgt+ ~~~ +0,A5 
z, = a M+dodst+-- +d,da, 


be any components of the vectors A‘Yo and AtZy. (If Zo=Yo; 


220 Proper Numbers and Proper Vectors of a Matrix 


y, and z, will be any two components whatever of the vector .44 Yy.) 
Let us form the expression 


YY, Yge1 
Zp Sgt 


(2) [¥e Z%] = 


On working this out, we find that 


(3) (Ye. Ze) = (¢ed) —eyde) (Ay — Ap) AAs 
+ (cg, —6,d3)(A, — Ag) ALAS + O(ASA8). 
Hence it follows that 


(Yeti, Ze+1] (3)' 
4 ane = AA + O| =) - 
© oz Ore 
In particular, one may take Zp>= AY , whereupon 
(5) (yes Zl = (ye yer) =| 7 FAY, 
Yit+1 Vere 


It can be verified that 


(6) (9 Yeti) = C162(A1 — Ag) 2A, aS +€¢3(A, — Ag) 2M 45 = 0(45,A5). 
Therefore 


[Yeti ¥ete] (#)' 
7 ae = 1,1.+0(—] - 
(7) [Yes Yeti] fet Ay 


The disappearance of significant figures which we alluded to is 
occasioned by the fact that the elements of determinant (2) and of 
determinant (5) as well become almost proportional to one another 
to the degree that the step of the iteration is increased. 

Observation. In case the first proper number is multiple, 
A, =Ag= +--+ =A,, but the elementary divisors corresponding to it are 
linear, the process indicated gives the product 4,A,4,. This process 
will generally give the product of the two “eldest”? roots of the 
minimum, and not of the characteristic, polynomial. Therefore in 
utilizing the iterative process, one can judge of the multiplicity of 
the dominant proper number only by means of indirect considera- 
tions. An indication of the order of the multiplicity can be given, 
for example, by a comparison of the trace of the matrix with the 
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computed magnitudes of the first and second roots of the minimum 
polynomial. 

It is obvious that by means of three components of the vector 
A‘Y, or by means of corresponding components of the vectors 
AkY,, A*+1 Yo, A4+2Yo, we can analogously construct determinants 
[ys 245 ¢,) or [Yg. Ye+1. Ygee]. The ratios of corresponding deter- 
minants will give us an approximate value for the product 4) A293. 
The process can, theoretically, be continued to the determination of 
the product of the k roots. Practically, the process yields only very 
rough values for the product of two or three roots. 

As an example we shall determine the second proper number for 
the matrix we examined in Paragraph 2, § 30. 

Adopting for y, the second component of the vector A‘Xo, and for 
z, the fourth component, we obtain 


8.3912886 8.4032694 


[ys 218] =| _ 9 3998078 —8,4089592 | = 9-0240!24- 


Analogously 
(¥19. Zip] = 0.0159949. 


Thus 2,42 ~ 0.6661 10. 

In § 30 we saw that 4,~1.001. This gives the value A. =0.6654. 
If we use the value refined by the scalar product method, 
A, = 1.000000, we will obtain as 4, the value 0.666, which is correct 
to an accuracy of one unit in the third decimal: 


(Ay = 3 = 0.66666...). 


If as y, the first component of the vector A4X, be adopted, and the 
second component as z,, we will obtain 


{¥18) Z1g] = 0.0397902 
[y19. Z19] = 0.0265541, 
This gives 
AyAo ~ 0.667353. 


Utilizing the refined value of A,, we obtain 2.~0.667. Finally 
we shall determine A. by the ratio of determinants of the form 


Ye Yt) 
Yat. Ygtez 
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where y, is any component of the vector A*X 9. Taking as y, the 
first component, we obtain 


1- 4.6760089 4.6779433 | _ 
418: Y19] = | 4 6779433 4.6792336| 


Analogously, 


—0,0030156. 


[¥i7s ¥ig}) = —0.0045170. 
Thus 4,4.~0.66761 and 4.~0.668. 


§ 32, DETERMINATION OF THE PROPER 
NUMBERS NEXT IN LINE AND THEIR 
PROPER VECTORS AS WELL 


The process of determining the proper numbers following after 
A, described by us in the preceding section, does not make possible 
the determination of the proper vectors belonging to these numbers. 
In this section we shall give devices that permit the determination 
not only of the proper numbers that follow A, but also of the proper 
vectors belonging to them. 
1. The A-difference. Let there be computed the sequence 


(1) Ys. Yo, ee so Yr e+e Yer 


and from it let A, ai be determined. Here y, is any component 
k 
of the iterate Y,=A*Y5. 


Let us form the difference 


(2) Yiti— Ay, = 62(Ao — Ay) As + cee + 6y(Aq— A) An 


If A, is greater in modulus than all the rest of the proper numbers 
and ¢.#0, the first term of this difference will predominate and we 
shall be able to determine A, in a way analogous to that by which we 
determined 4, viz.: 


(3) qo mw Yet —Ay, 

op AY ga 
However in such a determination of A, we encounter a disappearance 
of significant figures, since in the numerator and denominator of 
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ratio (3) we have to subtract quantities close to one another. It is 
expedient in practice, after finding A, from the ratio of y,+, and y,, 
to turn back and determine As from the ratio 


(4) Ae ~ Ymt) —AYn, 


m<k, 
Yn — AY¥n—1 


taking as m the least number for which the predominance of A, over 
the succeeding proper numbers has already begun to make itself 
felt. The suggested device gives for A, values that are rough, to be 
sure; however they are frequently adequate for practical purposes. 
Theoretically it is possible by means of an analogous process to 
determine the succeeding proper numbers too. 

It is obvious that to determine the second proper vector the process 
of forming the A-difference must be carried out on the sequence 
AYo, A®¥o,..., A*Yo,...; indeed, the difference 


AY, —A,A*¥q = ao(Ag—Ay)AsXot °° + +a,(A,—A,)ALX, 


shows that the components of the vector X, may be found in a 
manner analogous to that with which we determined the com- 
ponents of the vector X, in § 29. 

For an example, we determine the second proper number of the 
matrix discussed in § 30, Paragraph 2. 

As A, we shall take not only the value obtained directly from the 
ratios of the components of the 20th and 19th iterates (A, ~ 1.001), 
but also the value refined by means of the scalar product 
(A, ~ 1.000000). 

Adopting as y, the first component of the vector A‘ Yo, we obtain 
(for A,;~ 1.000000), taking into account the 17th, 18th and 19th 
iterates (m= 18): 


A w Yt — ALY mn me 4.677943 — 4.676009 -_ 0.001934 = 0 667] 
2 in MYm-1 4.676009 —4.673110 ~ 0.002899 ~ °!" 


Analogously, adopting as y, the fourth component of the vector 
A*tY,, we obtain 


_, 0.009131 


Aa = -- 0.013667 


~ 0.6681. 
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Thus knowledge of a sufficiently exact value of 1, has made it 
possible to determine A, also with sufficient accuracy (to three places 
after the decimal). (Exactly, 4,=0.666... .) 

If as A, we take the rougher value 4;~ 1.001, on computing the 
previous ratio we run up against the phenomenon described—the 
disappearance of significant figures. In this case as m a number 
considerably less than 20 must be taken. 

Thus, considering the 9th, 10th and 11th iterates of the vector 
APY, 

A®Y, Aloy, Ally, 
4.4665336 4.5365193 4.5841480 
7.1243407 7.540765 1 7.8281626 
0.0699857  0.0476287 0.0321941 


~7.4707539 —7.7678185 —7.9777169 
4.1901061 4.3570946  4.4667878, 


we obtain, on computing the values ¥,,41—A,y,,: 
m=9 m=10 
0.06552 0.04309 
0.40930 0.27986 
—0.02242 —0.01548 
—0,28959 —0.20213. 


The ratios of these quantitites give for A, the values 


0.658 

0.684 

0.690 

0.698. 
Thus knowledge of a value for A, that is very rough allows us none- 
theless to obtain for A, a value accurate to three units of the second 
place, by using the early iterates. 


The A-difference method permits the determination of the com- 
ponents of the second proper vector too. 
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For the matrix under consideration, the second proper vector has 
the components 


(1, 3% = 6.2, -2 = —0.333..., —f#b = —4.733...). 


Approximate values for the components of the second proper 
vector can be obtained as the respective ratios of the components of 
the vector A+! Yy—A,A™Yp. Taking m=9, we obtain, by utilizing 
the components of the vector 4!9°Yy—4,A®Yo, computed earlier, the 
following values for the components of the proper vector (after 
normalization) : 


(1.00; 6.49; —0.36; —4.69), 


which is in sufficiently close agreement with the exact value. 

2. The method of exhaustion. The method of exhaustion in the 
general casc requires not only knowledge of the first proper number, 
but also of the proper vectors that correspond to it for the matrix A 
and its transpose as well. 

Let us say that we have determined with sufficient accuracy 4), 
X, and X}, where the vectors X, and X; have been so normalized 
that their scalar product is equal to unity. 

Regarding the components of the vector X, as a column, and 
the components of the vector X| as a row, we form the matrix 
product 1,X}. This will be the square matrix 


. ’ go ¢ 
‘od 0 Cod 6 US B tat) MCCS OB fal 5 | 
td e 6 
MoiXy1  Xaixy1 + +s Kaikan} 
é 4 C 
Mni*i1  XyX21 ee A yiXny 


We remark that the matrix product XX, is a number, to wit the 
scalar product of the vectors 1; and X. 
Next form the matrix 


(5) Ay od A—A, XX}. 


We shall show that the matrix A, has the same proper numbers 
and vectors as the matrix A, with the exception of the first proper 
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number, in place of which there will be a proper number equal to 
zero. Indeed, 


A,X, = AX, —A,(X,X4)X, 


= AX, — A,X ,(X4X)) = AX, -A\ xX, = 0 
(6) 


= AX; ‘cole an dieains = A,X;, (¢#1) 


since X,X,=1, and X}X;=0, on the strength of the orthogonal 
properties of the vectors X),..., X, and X}4,..., Xp. 

The property of the matrix A, which we have indicated makes it 
possible to start from the vector sequence A,Yo, ..., A,"Yo,... and 
determine A, and X, in a manner analogous to that by which we 
determined A, and X, from the sequence AYp, ..., A* Yo, ..., since 
the proper number A, will be the first proper number of the matrix 
A,. We shall call this process the ‘ process of exhaustion”’. 

We shall show that 


(7) AY = A™Yo—Ai Xi X14, 


i.e., that in the practical application of the indicated process there 
is no need of computing the matrix A, in fact, or of forming the 
series of vectors A,Yo,...,A,™Yo,..., but that it is sufficient to 
compute just two adjacent vectors A,"+!Y,) and A,™Yo by formula 
(7). 

In order to establish equation (7) we shall introduce the so-called 
bilinear resolution of the matrix A. 

On the basis of the orthonormal properties of the system of proper 
vectors of the matrix A and its transpose, the following matrix 
equation is valid: 


1g.) CED OF earn © a eo 


Multiplying this equation on the left by 4 and replacing 4X; by 
A,X;, i=1,..., 2, we obtain 


(8) A = AX X{+AgXoX ot +++ $AXXy. 
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The process of exhaustion annihilates the first summand in this 
resolution, whence it follows that 


A, = haXoXat - == HAL XX: 


Moreover 
A™ = ATX XAG XoXo t+ 60+ FAX, X). 
Analogously 
AT = AZXaXa 4+ +++ + AX, X. 
Therefore 


whence issues equation (7). 

Thus for the practical application of the process of exhaustion, 
one has to compute the vector X,X1Yp, form the vectors A,"*'Yo 
and A,"Y, by formula (7), and then find A. and X. in a manner 
analogous to that by which we determined 4, and X, in § 29. It 
is obvious that in course of this all the devices for improving the con- 
vergence of the iterations which we have described in § 30 may be 
employed, in particular the 6? proccss. 

We note that one must take as m a number considerably less than 
the number of the iterate fram which the components of the first 
proper vector are determined. We note too that in case one finds 
it necessary to determine with great accuracy the second proper 
number and the vector belonging to it, one will find it in fact 
expedient to form the matrix 4d, and to compute its iterates. 

Example. Let us again consider the matrix of § 30, Paragraph 2. 

The method of exhaustion requires, for the determination of the 
second proper number, knowledge of the first proper number, as 
also of the proper vectors of the matrix A and of matrix A’ as well 
that belong to it. It is therefore necessary in utilizing this method 
to compute, along with the sequence of iterates A‘ Yo, that of A’* Yo 
also. Thus in determining 2, we must always refine the value 
obtained , by means of the scalar product method. 

In our example, using twelve iterations of the vector Yo 
=(1, 1, 1, 1) by the matrix 4 and by A’, we have obtained for 2, 
the value A, = 1.000000 (see § 30, Paragraph 2). 

For the components of the proper vectors of the matrices A and A’ 
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we obtain, after normalizing the components of the vectors .429Y5 
and A’20Y,, the values 

1.00000 1.00000 

1.79757 —0.00027 

0.00018 5.01676 

— 1.79838 —0.14256. 

(The exact values of the components of the first proper vector of the 
matrix A are (1, 1.8,0, —1.8).) Following the theory, it is first 
necessary to normalize the vectors X, and Xj; so that (X,, Xj)= 1. 
The normalizing factor computes out to be ¢=0.795678. Thus for 


the components of the first proper vectors of the matrices A and A’ 
we obtain the values 


1.00000 0.79568 

1.79757 —0.00021 

0.00018 3.99173 
— 1.79838 —0.11343. 


Now we can form the matrix product XX}; it is: 
0.79568 —0.00021 3.99173 —0.11343 
1.43029 —0.00038 7.17541 —0.20390 


0.00014 0 0.00072 —0.00002 
— 1.43093 0.00038 —7.17865 0.20399 


XX} aad 


Next we form the matrix 4,: 
0.20432 0.00021 —2.99173 0.11343 
—0.43029 0.77816 —6.84208 0.53723 


—0.00014 —-0.02525 0.55484 —0.02523 
1.43093 —0.88927 -—1.46579 —0.09288 


and form the iterates of the vector Yy=(1, 1, 1, 1) by this matrix. 


Determination of Proper Numbers Next in Line 229 


We adduce the 17th and 18th iterates of the matrix 4: 


AY, —AI*Y, 
—0.00869  —0.00580 
— 0.05388  —0.03597 

0.00290 0.00193 

0.04107 0.02741 


—0.01861 —0.01242. 


The ratios of the components of the 17th and 18th iterates give 
for As the values 


0.667 
0.668 
0.666 
0.667 . 


The ratios of the corresponding components of the 18th iterate 
give for the components of the second proper vector the values 


1.00 
6.20 
— 0.333 
—4.73 . 


We sec that not only the second propcr number A. but also the 
components of the second proper vector are determined more ex- 
actly by the method of exhaustion than by the A-difference method. 
However this method requires, for the case of a nonsymmetric 
matrix, much additional work. In the case of a symmetric matrix 
the method of exhaustion can be recommended. 

In computing the second proper number and the components of 
the second proper vector, one can utilize the modification of the 
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process that was described, whereby the vector AjY, is computed, 
but the iterations of the matrix A, are avoided, by the formula 


ALYo = A‘'¥Q—A Xi X15. 
Making the computation, we obtain 


4.67377 
8.40142 
0.00084 
— 8.40521 


XX 5 = 


Now we compute AtY, for k=9 and 10. This gives 


k=9 k=10 
— 0.20724 —0.13725 
— 1.27708 -0.86065 
0.06915 0.04679 


0.93446 0.63739 . 
Hence we find 4,: 


0.662 
0.674 
0.677 
0.682 


and from A}"Yo, the components of the proper vector: (1.00, 6.27, 
—0.34, —4.64). 

3. The reduction method. Given the matrix A, let the first proper 
number 4, and the proper vector X, =(x),...,,) belonging to it 
have been computed. Let us consider the matrix 


xy 0 eee 0 


e e¢# ee @  @  @  @ 
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We confirm without difficulty that 


— oO. 0 
x1 
xo 

pei = =e } 0 
_* 9g l 
*) 


The matrix P-!AP is similar to the matrix A, and the proper num- 
bers of both matrices are thus identical. 


But 
2109 ain 
A, — 
“1 x) 
0 an—- Za Go-822% 
- = 92—— aja as 
P-1\AP = mo 0, Mis 
x 
Q @Qoe—-—"a),0 a.——a 
n2 x 12 nan x In 
A, dy bi, 
0 
: B 
0 
Thus 


|P-1AP—Al| = (A,—A)|B—All]. 


Consequently the proper numbers of the matrix A will be A, and 
the proper numbers of B, a matrix of the (n—1)th order. If X; be 
so normalized that x, =1, we will have 


ao, —X941,, 


Ang —%_212 0 © > Ban — X21 y 
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For the determination of A, we must obviously construct a sequence 
BYo,..., BéYo,... and find A, as the ratio of any components of 
the constructed vectors. 

Furthermore let Y be a proper vector of the matrix B. The 


matrix P-'AP will then have a proper vector (). Let us deter- 
mine y}. 


We have 


Ay 439 +--+ @y 
0 (* 
B Y 


x ne oe “ar - a( ) 
BY Y/}- 


Equating the first components of this vector equation, we obtain 


AY + ayoYo+ +++ +44,9, = Ay, 
whence 


Finally the proper vector X, of the matrix A is determined by the 
formula 


yw 
x _ P| e xoy) + y@ 
bt a rc aan 
xy ty 


Example. Let us determine the second proper number and the 
components of the proper vector belonging to it, for the matrix of 
§ 30, Paragraph 2. 

As we have seen, the first proper number for this matrix has been 
determined to be A,~1.001 (§ 30, Paragraph 2),.and the com- 
ponents of the proper vector belonging to it to be (1, 1.79757, 
0.00018, — 1.79838) (§ 33). 
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For determining A. we compute the matrix B: 
0.77778 —1.46424 0.33333 
B= {| -0.02525 0.55538  —0.02525 
—0.88889 —6.84606 O.11111 


Next we form iterates of the vector Yy=(1, 1, 1) by the matrix B. 
We give the result of the 15th and 16th iterations: 


BISX, BBY, 
—0.09565 —0.06388 
0.00725 0.00484 
0.06339 0.04243 


—0.02502 —0.0166I1. 


We determine A, from the ratios of the components of the !6th 
and 15th iterates. This gives 
0.668 
0.668 
0.669 . 


We now determine the components of the second proper vector. 
With this in view, we determine, to begin with, 


- ByoYo + 21345 + 41444 0.00484 a _0.0145. 


y ish, ~ 20.333 
Next we compute the components xa, x3, x, of vector Xy: 
x» = —0.0900 
x3 = 0.00484 
x, = 0.0685. 


Thus the components of the vector X, are (—0.0145, —0.0900, 
0.00484, 0.0685), or, after normalization, (1, 6.21, —0.333, — 4.72). 

As we sec, the reduction method makes possible the exact deter- 
mination not only of the second proper number itself but also of the 
proper vector belonging to it. 
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§ 33. DETERMINATION OF THE FIRST 
PROPER NUMBER. SECOND CASE 

Let us now consider the case where the elementary divisors of 
the matrix are linear, but the first proper number /, is complex. 
In this case 4, will also be a proper number, so that of the proper 
numbers that are of greatest modulus there will be not less than two. 
Let us assume that |A3|<|A,|.. Let X, and Xz. be the proper vectors 
belonging to the proper numbers A, and ds. On the strength of 
§ 3, Paragraph 6, one may consider their components to be complex 
conjugates. Furthermore, if 


Yo = aX; +a .Xo+ a8 s +a,X, 


is a real initial vector, a, and a, will be complex conjugate numbers. 
Reasoning just as in § 29, we form the sequence of vectors 


(1) AY o,..., A*&YVo,... 


Any component of the vector Y,=A‘Y, will obviously have its 
previous form, 


(2) Y_ = CAL + coda t +++ +eyAny 
where ¢, and ¢y are complex conjugates. Let 


C¢,; = Reta, cy = Re-ie, 


3 

©) A, = reo, Ay = re7t0; 
then 

(4) ¥ = 2Rr* cos (KO +0) +egAgt +: 


The presence of the factor cos(k@+«) will be the cause of a marked 
oscillation of the values of y,, in magnitude and in sign as well. Thus 
the presence of complex roots that are the largest in modulus will at 
once be revealed in forming the iterates. Since r2=A,A,, we can 
determine the quantity r? by means of the determinants introduced 
in §31. Here, as distinct from the case of real roots, we will not 
encounter the disappearance of significant figures, so that r? is deter- 
mined with sufficient accuracy from the ratio of these determinants. 
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We shall show that having computed r, one can determine cos 0, 
utilizing the sequence ¥), Yo,-.. 5 ¥y,--- for this purpose. 
We introduce the expression! 


(5) Hye = Bryer tr yg). 
a\é 
We have from (4), accurately to terms of the order of (3) , that 
i 
y, ~~ 2Rrt cos(ké +a). 
Furthermore, since 


cos((k + 1)0+a]+cos[(k-—1)0+a] = 2 cos(kO+«) cos 0, 


we obviously have 


(6) cos 0 = Ft, 


UT 


; ee A. k 
with an accuracy to quantities of the order of (3) . 


Ay 
Since the case of complex roots is of little practical interest, we 
shali not dwell on the determination of the proper vector. For the 
same reason an illustrative example is not given. We have touched 
upon this case as upon the cases considered in the preceding section, 
only in order to show the possible peculiaritics of the course of the 
iterative process. 


§ 34. THE CASE OF A MATRIX WITH NON- 
LINEAR ELEMENTARY DIVISORS 


In all the preceding sections devoted to iterative methods, it has 
been assumed that the elementary divisors of the matrix arc linear, 
i.e., that the matrix A is reducible to diagonal form. In this section 
we abandon that requirement. As has already been noted in the 
introduction, the iterative process depends essentially on the struc- 
ture of the Jordan canonical form that is connected with the given 
matrix. We shall not consider the general case, but shall show ina 
most simple example only the character of the changes that occur if 
the matrix has nonlinear elementary divisors. 


1 A. Aitken [l). 
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Let us concern ourselves with the determination of the first proper 
number A, of the matrix A. We shall discuss the case in which 2, 
% 7 ) in the Jordan canonical 
form, and the next proper number A, is less than 2, in modulus. To 
simplify the calculation we shall consider the elementary divisors 
that correspond to the rest of the proper numbers to be linear. 

As previously, we adopt an arbitrary vector Yo and form the 
sequence AYo, A?Yo,..., Aé¥o,... . Weshall elucidate how any 
component y, of the vector Y, = A‘ ¥g depends on the corresponding 
powers of A; in such a case. Since the proper vectors of the matrix 
do not in this case form a basis, let us take as X,,..., X,, the vectors 
of that basis relative to which the linear transformation with the 
matrix A is brought into canonical form. Then 


is real and belongs to the box ( 


AX, = AX + Xs 


AX, = A, Xo 
(1) AX; = A3X3 
AX, = 1,X,. 
Let 
(2) X= (Xj Fajr eer Xaglo Ve = (Yrs +++ > Yat)s 
and 


Yo = aX, +aeXot eee +a,X,. 
We have, on the strength of (1): 


(4) AY = GA, X, +0,Xot+ andy Xo + aglgX3 t+ eee +a,A,4, 


e @e¢ @ @®@ @® $@® @© $@  @  @® @  @  @  @  @  @  j@ @ @  @®@  @  @®  @® « 


(5) A’¥, = a,AtX, + aykAt Xn tad Xo +agdaXgt +++ +4,44X,. 


Furthermore, in view of (2), we have 


Yun = a, Atx;; + (a,Aq 1k + agd})x;0 + 49A9x;5+ se +0,A8% 0 
= (4)x;) + oX;0)Aj +04x,;0kAy | + @5%;5A5 + see 4+ 4,%; As. 
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Thus any component of the vector Y, will have the form (we omit 
the first index, as before): 


(6) yy = CyAR +egkAl ! +egdht --- +c,ak. 
Yat 


Yt 
geometrical progression, owing to the presence of the factor & in the 


second term, to wit: 
Hert '\), 
Ye ali +0(;)| 


To determine A, from the ratio ae becomes practically im- 
k 


The ratio tends to 2, as before, but more slowly than any 


possible. 

We note that if the box to which A, belongs in the Jordan canonical 
form has a more complex structure, other powers of 4, also appear 
in expression (6); they are multiplied by the corresponding binomial 
coefficients: 


BR ge ey. 


r 
5 > +6,An- 


y= ave +k! +3 


Yat 


The ratio + approaches A; still more slowly. 


In Sener a slow convergence of the process provides grounds for 
surmising that a nonlinear elementary divisor, connected with the 
largest proper number, is present. 

However the process by means of which the product of two proper 
numbers is determined will converge rapidly enough in the case in 
hand. It can be confirmed without difficulty that 


k 
7 (Yitrs Yero] — 22 +0(3) > 
”) [Yar Yetr] A 
for indeed on computing [y,, yg+1] = Ye Het) We find that 
Yeti Vere 


k 
[¥ Yeril = 63 at +0(F Z): whence (7) follows. 


Ratio (7) makes possible the determination of A). 
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We shall now show how, in case A, belongs to the box ( . , 
1 


and this circumstance is suspected from the slow convergence of the 
sequence ¥,, one can satisfy oneself that this is indeed the fact. 
Let us have found 4, from the ratio (7). Let us construct 


Ay, = yeri— Arye = C1Ay*! +eo(kK+1)AL tegdg + + - 


(8) +c,Agt} Sea, —cokAh —c3AsA; ns —c Ata, 
= t9A} +-O(A5). 
Then 
Ayntr _ A3\* 
Bae = h+0(7) > 
1.€.; Sees tends to A, fast enough. The coincidence of the limit of 
Ay,+1 


Ay with the value for A, computed earlier, and the fact of the 
i 


Ay. +1 
Ay, 
that A, figures in a box of the second order. 
Instead of the computation described, one can have recourse to 
computation of the second A-difference. 
If A, belongs to a box of the second order, the second A-difference 


rapid convergence of to A, serve as confirmation of the surmise 


Aty, = Ayysy,—A\Ay, = 0(A3) 


will be small in comparison with the component y, itself. 

The case of nonlinear elementary divisors of the matrix is very 
rarely met with in practice. We shall therefore not dwell either on 
the determination of the proper vector that belongs to the first 
proper number, or on the determination of the succeeding proper 
numbers and the vectors belonging to them. 

We remark only that all the devices we have discussed for the case 
of linear elementary divisors may be employed in the general case 
too, after suitable modification.! 


1 A. Aitken [1]; K. A. Semendiaev [1). 
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§35. IMPROVING THE CONVERGENCE OF 
THE ITERATIVE PROCESS FOR SOLVING 
SYSTEMS OF LINEAR EQUATIONS ' 


As we saw in § 17, in solving a system of linear equations given 
in the form X= AX+F by the method of iteration, we consider that 
Xz XH =AX*-4+F, Here 


(1) X— Xb) = (Xb — XH) + (NED — XUV) poe, 


Let us calculate approximately the sum of this series. 
We have 


(2) Xe Xb) = A(X XUV) see. = AMX — XO), 


Let A,,..., A, be the proper numbers of the matrix A; we shall 
consider them to be distinct, for simplicity, and to be arranged in 
order of diminishing moduli, and shall consider U,, Us,..., U, to 
be the proper vectors that correspond to them. Let us resolve the 
vector X() — X¥ (©) in terms of the proper vectors of the matrix A: 


X'1)— X (0) = a,U,+ eae +a,U,,. 


Then, on the strength of (2): 
X®V_ KO = MU +--+ +0,AU, = «AU, + 0(A4). 


As was shown in § 29, we can determine the first proper number 
A, from the ratio of the components of the vectors (X +) — X) and 
(X (4) —X%-D), with sufficiently large k. Moreover 


(X etl) — X(b)) + (Xb+2) — Xt) po. 
(3) = a AVL +A, +ait «JU +--+ a,b ed4 a+ --)U, 


k k (A4-1) (k) 
ieee ee 


We have the right to sum the geometrical progressions, for all 
|A;| <1, since we consider the iterative process to be convergent. 


3 L. A. Liusternik [1]. 
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Thus, accurate to quantities of the order of Ai, we have, as the solu- 
tion of the linear system, the expression 


Nlktl) — Yb) 


~ (k) 
(4) Xa Xb) + [=7, 


The extra term in equation (4) is effortlessly computed, and greatly 
improves the convergence of the iterative process. 
If the solution of the linear system is approximately determined by 


£ 

means of the expression Xz 5 A/F, then, reasoning analogously, 
r= ; 

we find that 


ASIP 
=r 


k 
X~ > AIF+ 
is 


will give a better approximation to the sought solution. Here the 
quantity 4, is found approximately from the ratio of the com- 
ponents of the vectors A4+1F and AéF. 
Thus for the example analysed by us in §17, we obtained (see 
§ 29), on proceeding from the 14th approximation, A, = 0.4800. 
Furthermore, from Table XIII, § 17, we have 


X43) = (1.53490847, 0.12200958, 1.97509985, 1.41289889). 


Abt ; 
On computing Toa we obtain 
Ay 
Ab-lF 
= (0.00005656, 0.00005656, 0.00005656). 
Ai 


Thus 
X ~ (1.534965, 0.122010, 1.975156, 1.412955). 


We see that, with accuracy to the sixth figure, the improved solution 
coincides with that found by Gauss’s method. 
Moreover, 


6 
> AW = (1.52533, 0.12201, 1.96551, 1.40333). 
i=0 
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Having taken A, =0.4792 (see § 29) and used the device for im- 
provement we find that 


6 
S apy AF 


0 + 05008 ~ (1.53495, 0.12203, 1.97514, 1.41293). 


As we see from Table XIII of § 17, the approximation obtained 
coincides better with the exact solution than does the [4th approxi- 
mation. 

The method for improving the convergence here suggested may 
also be applied to the Seidel method, for that method, applied to 
the system X=AX+F, is equivalent to application of the usual 
iterative method to the system 


X = A\X+F, where «4, = (J-—B)-1C. 


Here 8 and C are the triangular constituent matrices of the 
matrix A (see § 19). The largest proper number 4, of the matrix 
A, is determined from the ratio of the components of the vectors 
X(k4+1) — Xk) and X (4) — X'-)), and for improving the convergence 
one can as before utilize the formula 


X (kel) — Xk) 


Xx Xb + 
1 — fey 


In conclusion we make mention of methods that have quite 
recently appeared in the literature, which are based on the ideas of 
functional analysis. These methods, developed for the solution of 
more general problems of operational calculus, give, in particular, 
a solution of the basic problems of linear algebra (not only the 
solution of a linear algebraic system, but also the finding of the 
proper numbers of a matrix), if for the operator under study one 
adopts the linear transformation connected with the given matrix. 

The reader who is interested in these methods and who is familiar 
with the basic ideas of functional analysis is referred to the extensive 
article of L. V. Kantorovich [1], which contains a large biblio- 
graphy, and also to another work of the same author, [2], as well 
as to those of I. P. Natanson [1], M. Sh. Birman [1], and M. K. 
Gavurin [1]. 
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